Wearable electronic glasses that move a virtual object in response to movement of a field of view

ABSTRACT

A method detects a real object with a wearable electronic device and displays a virtual image of the real object with the real object on a display of the wearable electronic device. The wearable electronic device displays movement of the virtual image of the real object to show a task to be completed and detects a completion of the task. The wearable electronic device moves, in response to the virtual image of the real object no longer being within a field of view of the wearable electronic device, the virtual image of the real object to an opposite side of the real object such that the virtual image of the real object is within the field of view.

BACKGROUND

Electronic devices can provide users with digital content that enhancesa view of the real world. These enhancements include virtual images thatprovide users with visual information. For example, augmented realityuses computer-generated elements to supplement a perception of thephysical world that viewers perceive.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a computer system in accordance with an example embodiment.

FIG. 2 is a method to provide instructions from an electronic device toa wearable electronic device in accordance with an example embodiment.

FIG. 3 is a method to display instructions through a wearable electronicdevice to instruct a wearer of the wearable electronic device to performan action in accordance with an example embodiment.

FIG. 4 is a method to display a virtual object with a real object anddetect completion of a task on the real object in accordance with anexample embodiment.

FIG. 5 is a method to maintain a consistent distance between a virtualobject and a real object that are in a field of view of a wearableelectronic device in accordance with an example embodiment.

FIG. 6 is a method to move a location of a virtual object in response tomovement of a wearable electronic device and/or a real object inaccordance with an example embodiment.

FIG. 7 is a method to display, with and/or through a wearable electronicdevice, actions to virtual objects received from an electronic device inaccordance with an example embodiment.

FIG. 8 is a method to display a translation of a writing on a realobject in accordance with an example embodiment.

FIG. 9 is a method to display virtual content associated with an area oran object when a wearable electronic device views the area, views theobject, and/or comes within proximity of the area and/or the object inaccordance with an example embodiment.

FIG. 10 is a method to provide instructions to a wearable electronicdevice for how to cure an error with an electronic device in accordancewith an example embodiment.

FIG. 11 is a method to share a location of a virtual object withmultiple wearable electronic devices in accordance with an exampleembodiment.

FIGS. 12A-12D show a computer system or electronic device system thatprovides instructions to a wearable electronic device in accordance withan example embodiment.

FIGS. 13A-13C show a wearable electronic device that provides a virtualobject with a real object in accordance with an example embodiment.

FIG. 14 shows a wearable electronic device that provides a virtualobject located on a real object in accordance with an exampleembodiment.

FIGS. 15A-15C show a computer system or electronic device system thatprovides instructions to a wearable electronic device in accordance withan example embodiment.

FIGS. 16A-16C show a wearable electronic device that provides a virtualobject with a real object in accordance with an example embodiment.

FIGS. 17A-17D show a wearable electronic device that provides a virtualobject with a real object in accordance with an example embodiment.

FIGS. 18A-18C show a wearable electronic device that translates writingon a real object in accordance with an example embodiment.

FIG. 19 shows a computer system or electronic device system thatincludes wearable electronic devices that communicate with each otherover a network in accordance with an example embodiment.

FIG. 20 shows a computer system or electronic device system thatincludes wearable electronic devices that communicate with each otherover a network in accordance with an example embodiment.

FIG. 21 shows a computer system or electronic device system thatincludes wearable electronic devices that communicate with each otherover a network in accordance with an example embodiment.

FIG. 22 is a computer system or electronic device system in accordancewith an example embodiment.

SUMMARY OF THE INVENTION

One example embodiment is a method that detects a real object with awearable electronic device and displays a virtual image of the realobject with the real object on a display of the wearable electronicdevice. The wearable electronic device displays movement of the virtualimage of the real object to show a task to be completed and detects acompletion of the task.

DETAILED DESCRIPTION

Example embodiments include systems, apparatus, and methods that includeelectronic devices that utilize digital content to supplement a view ofthe real world. This content includes digital images, virtual images,computer-mediated reality, augmented reality, virtual reality, and othertechnologies that supplement or alter the view of the real world.

FIG. 1 is a computer system 100 in accordance with an exampleembodiment. The computer system includes servers 110, storage 115,electronic devices 120, wearable electronic devices 125, an imagerysystem 130, a computer system 135 (such as the computer system 2200shown in FIG. 22 or components therein), peripheral devices 140, andwearable electronic glasses 145 in communication with each other throughone or more networks 150. Blocks and methods discussed herein executewith the computer system or one or more of the electronic devices,servers, and/or components therein.

The servers 110 include one or more of a processor unit 160 with one ormore processors and computer readable medium 162 (CRM), such as randomaccess memory and/or read only memory. The electronic devices 120include one or more of a processor unit 170, CRM 172, and a display 174.The processing unit communicates with the CRM to execute operations andtasks that implement or assist in implementing example embodiments.

By way of example, the electronic devices include, but are not limitedto, handheld portable electronic devices (HPEDs), wearable electronicdevices, wearable electronic glasses, portable electronic devices,computing devices, electronic devices with cellular or mobile phonecapabilities, digital cameras, desktop computers, servers, portablecomputers (such as tablet and notebook computers), handheld audioplaying devices (example, handheld devices for downloading and playingmusic and videos), personal digital assistants (PDAs), combinations ofthese devices, devices with a processor or processing unit and a memory,and other portable and non-portable electronic devices and systems.

By way of example, the networks 150 include one or more of the internet,an intranet, an extranet, a cellular network, a local area network(LAN), a home area network (HAN), metropolitan area network (MAN), awide area network (WAN), public and private networks, etc.

By way of example, the storage 115 can include various types of storagethat include, but are not limited to magnetic storage and opticalstorage, such as hard disks, magnetic tape, disk cartridges, universalserial bus (USB) flash memory, compact disk read-only memory (CD-ROM),digital video disk read-only memory (DVD-ROM), CD-recordable memory,CD-rewritable memory, photoCD, and web-based storage. Storage caninclude storage pools that are hosted by third parties, such as anoperator of a data center. The electronic devices, servers, and/or othercomponents can use the storage to store files, software applications,data objects, etc. Storage can be accessed through a web serviceapplication programming interface, a web-based user interface, or othermechanisms.

FIG. 2 is a method to provide instructions from an electronic device toa wearable electronic device.

Block 200 states capture images with a wearable electronic device of auser.

By way of example, two-dimensional (2D) and/or three-dimensional (3D)images are captured with an optical device of a wearable electronicdevice, such as with a camera or lens. The images can be recorded,stored in memory, displayed, transmitted, and/or provided to a user. Forinstance, video images and audio are captured with the wearableelectronic device and then wirelessly transmitted over a network.

Block 210 states display the images in real-time on an electronic devicethat is remote from the wearable electronic device.

The remote electronic device receives the captured images from thewearable electronic device and displays or projects these images. Forinstance, the images are displayed on or through a display or presentedas a three dimensional image, such as a volumetric display device, 3Ddisplay, or 3D projection. The electronic device and the wearableelectronic device can be physically remote from each other, such asbeing in different cities or states, being at different locations in abuilding, being in different rooms, etc.

Block 220 states receive, at the wearable electronic device and from theremote electronic device, instructions that provide a task for the userto perform.

The remote electronic device receives instructions from a user and/or anelectronic device. For instance, a user interacts with a user interface(UI) or graphical user interface (GUI) to provide commands and/orinstructions that are transmitted to or provided to the wearableelectronic device. These commands and/or instructions provide a task forthe user or wearer of the wearable electronic device to perform.

Block 230 states display, through and/or with a display of the wearableelectronic device, the instructions received from the electronic device.

The wearable electronic device receives the instructions from the remoteelectronic and then displays, stores, or provides these instructions.For instance, the instructions are displayed or provided to a wearer ofthe wearable electronic device.

Consider an example in which a person wears a pair of wearableelectronic glasses (such as GOOGLE glass). These glasses capture videoimages of a field of view of the user and transmit the video images to ahandheld portable electronic device (HPED) that is remotely located fromthe glasses. The video images appear on a display of the HPED so a userof the HPED can view what the person sees through the glasses. The userinteracts with the HPED and provides instructions or information to theperson wearing the glasses. By way of example, the instructions includeone or more of voice commands, text commands, images, or interactionswith controls. These instructions transmit to the wearable electronicglasses so the person wearing these glasses can view or receive theinstructions. For example, the instructions include images in augmentedreality or virtual reality that the wearer of the glasses views.

Consider an example in which a person located in the state of Ohio wearsa pair of wearable electronic glasses that transmits images in real-timefrom the wearable electronic glasses to a personal computer of anotherperson located in the state of California. The personal computercaptures or assists in capturing 3D images of the person located inCalifornia and then transmits these images to the wearable electronicglasses such that the 3D images appear through the glasses to the personlocated in Ohio. These 3D images include information or actions thatinstruct the person located in Ohio how to proceed, act, or move.

FIG. 3 is a method to display instructions through a wearable electronicdevice to instruct a wearer of the wearable electronic device to performan action.

Block 300 states display, on an electronic device, controls that provideinstructions for a wearer of a wearable electronic device.

The electronic device includes a display with controls that enable auser of the electronic device to provide or generate instructions forthe wearer of the wearable electronic device. For example, the controlsappear on or are generated by the display of the electronic deviceand/or on the electronic device.

For instance, a user interacts with a user interface on the electronicdevice to create instructions, commands, and/or actions for the wearerof the wearable electronic device. The electronic device and thewearable electronic device can be physically remote from each other,such as being in different cities or states, being at differentlocations in a building, being in different rooms, etc.

Block 310 states receive, at the electronic device and through thecontrols, instructions that instruct the wearer of the wearableelectronic device to perform an action.

The electronic device receives or generates the instructions thatinstruct the wearer of the wearable electronic device. For instance, auser of a remote electronic device interacts with its display or a userinterface to generate the instructions. As another example, theelectronic device includes one or more sensors that sense an action ofthe user and generates from the sensed action instruction for the wearerof the wearable electronic device.

Block 320 states transmit, from the electronic device to the wearableelectronic device, the instructions that instruct the wearer of thewearable electronic device to perform the action.

By way of example, the instructions are transmitted over one or morenetworks from the remote electronic device to the wearable electronicdevice. The instructions can be stored in memory and then provided tothe wearable electronic device.

Block 330 state display, through and/or with the wearable electronicdevice, the instructions that instruct the wearer of the wearableelectronic device to perform the action.

The wearable electronic device receives the instructions and thenprovides or displays these instructions to the wearer of the wearableelectronic device. For instance, the instructions are projected by ordisplayed through a glass, camera, or lens of the wearable electronicdevice or another electronic device. These instructions are visible tothe wearer of the wearable electronic device.

Consider an example in which a wearer wears a wearable electronic devicethat captures video and transmits this video to a display of a tabletcomputer. A user of the tablet computer is thus able to see what thewearer of the wearable electronic device sees since a field of vision ofthe wearer is captured with the wearable electronic device andtransmitted to the tablet computer. The display of the tablet computeralso includes controls (such as GUI controls) that enable the user ofthe tablet computer to generate instructions or commands. Theseinstructions are transmitted back to the wearable electronic device andprovided to the wearer. The wearer views the instructions through a lensof the wearable electronic device and acts according to the informationprovided from the user of the tablet computer. For example, the wearerfollows the instructions viewed with the wearable electronic device.

Consider an example in which a remote electronic device includes a 3Dmotion control system or gesture controlled system that receivescommands from a user. The user performs gestures with his hands and armsto generate commands and/or instructions. These commands are transmittedto and displayed to a wearable electronic device as instructions orinformation to the wearer of the wearable electronic device. Thecommands from the user are replicated at the wearable electronic device.For instance, hand and/or body movements of the user of the remoteelectronic device are recorded and then displayed as 3D images orvirtual images to the wearer of the wearable electronic device.Alternatively, the hand and/or body movements of the user function asinstructions or commands that are transmitted to or relayed to thewearable electronic device. For example, a hand gesture in one directionand with a certain motion signifies to the wearer of the wearableelectronic device to walk in a particular direction. A hand gesture inanother direction and with another motion signifies to the wearer of thewearable electronic device to stop and sit down. These gestures canappear as virtual hand gestures, the actual hand gestures, or in anotherform, such as text instructions, audio instructions, or instructionsthat use icons, symbols, or graphics.

Consider an example in which a first user at a first geographicallocation wears a first pair of wearable electronic glasses, and a seconduser at a second geographical location wears a second pair of wearableelectronic glasses. A field of vision of the second user transmits tothe first user such that the first user sees through the first wearableelectronic glasses images at the second geographical location that thesecond user sees. The first user makes hand gestures into his field ofvision being generated at the first wearable electronic glasses. Thesehand gestures transmit to the second user and appear as virtual 3Dimages in the field of view of the second user at the secondgeographical location. The second user then emulates or copies thesevirtual 3D hand gestures with real hand movements from his own hands.For instance, the hand gestures instructed or showed the second user topick up an item located at the second geographical location. The seconduser emulates the received hand gestures with his own hands orinterprets the hand gestures to signify an action to take.

A user at the remote electronic device is able to control mental and/orphysical actions of the wearer of the wearable electronic device. Theuser at the remote electronic device generates or produces commands orinstructions for mental and/or physical actions, and these commands orinstructions are provided to the wearer of the wearable electronicdevice. The user is thus able to control or instruct the wearer how toact. These mental and/or physical actions include, but are not limitedto, thinking, calculating, envisioning, dreaming, walking, running,standing, sitting, moving in a particular direction, moving at a certainspeed, holding an object, throwing an object, speaking, moving aparticular body part (such as rotating a head or moving an arm or hand),jumping, grasping or retrieving an object, releasing an object, orperforming other physical or mental actions that humans are capable ofperforming.

Consider an example in which an electronic device communicates with ahead-mounted display in an augmented reality system. A user of theelectronic device provides instructions that instruct a wearer of thehead-mounted display how to act or move. By way of example, theseinstructions inform the wearer to walk in a particular direction, followanother individual, and perform a series of work-related tasks. Theinstructions are displayed to the wearer as virtual images that augmenta real-world view of the wearer.

FIG. 4 is a method to display a virtual object with a real object anddetect completion of a task on the real object.

Block 400 states detect a real object that is provided with the wearableelectronic device.

The wearable electronic device and/or another electronic device candetect the real object. For example, the real object is seen with orvisible through the wearable electronic device and/or detected with,captured with, or provided by the wearable electronic device.

Block 410 states display, through and/or with the wearable electronicdevice, a virtual object with the real object.

For example, the virtual object is displayed on a display of thewearable electronic device or provided by or seen through the wearableelectronic device. The virtual object exists adjacent to, on, or withthe real object. For instance, the virtual object is simultaneously orconcurrently provided with the real object such that the virtual objectexists next to, near, overlapping with, over, or on top of the realobject.

A size, a shape, and/or a color of the virtual object can emulate orcopy a size, a shape, and/or a color of the real object. The virtualobject looks like or has a similar physical appearance of the realobject. For instance, the virtual object emulates a physical appearanceof the real object or is a copy of the real object. Alternatively, thevirtual object can be different than the real object.

Block 420 states display, through and/or with the wearable electronicdevice and while the virtual object is with the real object, movement ofthe virtual object to show a task to be completed on and/or with thereal object.

The virtual object moves to show or illustrate current, intended, orfuture movements of the real object. These movements of the virtualobject can occur before the movements of the real object to show a taskor an action to be completed or performed on and/or with the realobject. For instance, the virtual object executes a series of movementsor operations, and the real object subsequently emulates the movementsor operations of the virtual object. The virtual object can provideinstructions for, to, or with the real object. For instance, suchinstructions illustrate or provide information for what actions shouldbe taken on or with the real object. Further yet, such instructions arenot limited to actual movements of the virtual object and/or the realobject but include sound (such as spoken words), text (such as writtenwords or characters), graphics, drawings, color, images, projections,light, etc.

Block 430 states detect, while the virtual object is with the realobject, completion of the task on the real object such that the movementof the real object uses and/or emulates the movement of the virtualobject.

The wearable electronic device and/or another electronic device candetect or determine when or if the real object completed or performedthe instructed task. For instance, the real object completes the taskwhen the real object performs one or more movements that copy or emulatethe one or more movements of the virtual object. As another example, thereal object completes the task when the real object follows instructionsprovided by the virtual object.

Consider an example in which a user wears a pair of wearable electronicglasses. The user stands next to and views a computer printer (i.e., areal object) through a lens of the wearable electronic glasses. The userwould like help with operating the printer and requests virtualassistance. In response to this request, a virtual image of the computerprinter appears in the field of view of the user. This virtual image isa virtual printer that looks like the make and model of the realcomputer printer of the user. For instance, a display of the wearableelectronic glasses provides or projects the virtual image of thecomputer printer next to the actual or real computer printer. Thevirtual printer moves or provides instruction in response to a queryfrom the user. For example, if the real printer has a paper jam, thenthe virtual printer shows operation of a series of steps to fix thepaper jam. For instance, the virtual printer shows a sequence of stepsas follows: opening its front paper tray; removing paper stuck in thistray; closing the front paper tray; opening a rear or back cover;removing paper jammed in rollers; and closing the rear or back cover.The virtual printer thus provided instruction and assistance on how tocure an actual paper jam in the computer printer of the user.

In this example, the computer printer detects a location of the paperjam, retrieves the appropriate virtual image sequence, and provides thissequence to the wearable electronic glasses of the user. After each stepof this process, a determination is made as to whether the task showedby the virtual object is performed on the real object. When the task iscorrectly performed on the real object, an indication is provided to theuser (such as a verbal or visual indication that the user correctlyperformed the task). When the task is not correctly performed on thereal object, an indication is provided to the user (such as a verbal orvisual indication that the user did not correctly perform the task).

When a task is correctly performed on the real object, the virtualobject proceeds to the next sequence. For example, while the virtualprinter is displayed next to the real printer in the field of view ofthe user, the virtual printer opens its front paper tray. In response tothis action, the user opens the front paper tray of the real printer.The real printer and/or the wearable electronic glasses determines asuccessful completion of this action, provides an indication of thissuccessful completion to the user, and then proceeds to the next step inthe sequence of curing the paper jam. For instance, the virtual printernext shows a virtual arm or person removing paper stuck in the frontpaper tray.

Consider another example in which a golf instructor wears a pair ofelectronic glasses while giving a golf lesson to a student. During thegolf lesson, the golf instructor views and records a golf swing of thestudent with the electronic glasses and requests a virtual golf swing ofa professional golfer to visually compare with the golf swing of thestudent. The recorded golf swing of the student and the virtual golfswing of the professional golfer are simultaneously played on or througha display of the electronic glasses so the golf instructor can comparethe golf swing of the student with the golf swing of the professionalgolfer. This visual side-by-side comparison reveals a flaw in the swingof the student. The golf instructor assists the student in changing thegolf swing to cure the flaw. The visual side-by-side comparison can alsobe saved or transmitted, such as being sent to the student to watch on apair of electronic glasses or projected in an augmented reality so thestudent can view the side-by-side comparison.

Consider an example in which a user desires assistance in finishing anassembly of a robotic device. The user wears electronic glasses, looksat the partially assembled robotic device, and retrieves a program toassist in finishing the assembly. The program produces athree-dimensional line drawing of the robotic device and overlays orsuperimposes this line drawing over the actual partially assembledrobotic device. This virtual overlay reveals that the robotic device ismissing an arm component. A virtual arm component appears and becomesfastened to the virtual overlay. The user is able to see how the virtualarm component attaches to the robotic device. With this information, theuser retrieves the actual arm component and attaches it to the actualrobotic device as previously shown in the movements of the virtualoverlay. The electronic glasses include sensors that confirm that thearm was successfully attached to the robotic device.

Consider an example in which a user cannot find a particular clothingstore while shopping in a large mall. The user wears a pair ofelectronic glasses and requests assistance in finding the clothingstore. A virtual image of a person appears in a field of view of theglasses, and this virtual person walks in a direction toward theclothing store. The user follows the virtual person down an escalator,past a food court, and toward an entranceway where the retail store islocated. When the user reaches the retail store, the virtual persondisappears from the view of the electronic glasses. Thus, the virtualperson walked with and/or guided the person to the clothing store.

Consider an example in which a student is studying physics from atextbook that is linked to a pair of electronic glasses. The studentreads the textbook while wearing the glasses. The glasses execute aprogram that tracks a location of where the student is reading in thetextbook and automatically retrieves visual examples to assist thestudent in understanding the material. For instance, the textbookillustrates a two-dimensional image of an atom. This image, however,appears as a three-dimensional image when viewed with the glasses.Further, the glasses provide video and other images that assist thestudent in learning physics. This information is displayed through theglasses to the student while the student is reading from the textbook.Virtual examples being displayed to the student coincide with thecurrent subject matter that the student is reading from the textbook.

FIG. 5 is a method to maintain a consistent distance between a virtualobject and a real object that are in a field of view of a wearableelectronic device.

Block 500 states display, through and/or with a wearable electronicdevice, a virtual object with a real object.

The virtual object can appear next to, above, below, besides, on, over,near, superimposed on, overlaid with, or partially on the real object.For instance, while the real object is visible with or in the field ofview of the wearable electronic device, the virtual object also appearsin the field of view. Thus the virtual object and the real objectsimultaneously or concurrently appear in the field of view of thewearable electronic device.

Block 510 states maintain a consistent distance between the virtualobject being displayed and the real object visible with the wearableelectronic device while the wearable electronic device and/or the realobject moves and alters a field of view that is viewable with thewearable electronic device.

By way of example, the virtual object is positioned with the real objectwhile the real object is in the field of view of the wearable electronicdevice. A user or wearer of the wearable electronic device is able tosee both the real object and the virtual object at the same time. Thereal object is seen through the wearable electronic device while thevirtual object is displayed to appear next to the real object.

A distance or separation can exist between the real object and thevirtual object being displayed with or viewable with the wearableelectronic device. This distance remains constant or consistent whilethe wearable electronic device and/or the real object moves and changesa location of the real object in the field of view of the wearableelectronic device.

Consider an example in which a user wears a pair of wearable electronicglasses and views a tablet computer that sits on a table. The wearableelectronic glasses generate and display on its lens a second tabletcomputer next to the real tablet computer. This second tablet computerdoes not actually exist on the table but is a virtual object thatappears to exist on the table. Both of the tablet computers appear in afield of view of the wearable electronic glasses and thus aresimultaneously visible to the user. A distance of about four inchesseparates the virtual tablet computer from the real tablet computer.When the user moves his head and therefore moves a position of thewearable electronic glasses, the distance between the two tabletcomputers remains consistent (i.e., remains at four inches). Thisdistance remains consistent since a location on the lens of where thevirtual tablet computer is located changes. If this location did notchange, then the virtual tablet computer would move in synchronizationwith movements of the wearable electronic glasses. As such, the locationon the lens of the virtual tablet computer adjusts to offset themovement of the wearable electronic glasses.

Continuing with this example, the user looks at the table such that thevirtual and real tablet computers appear in a center of a field of viewof the user. While the user moves or rotates his head twenty degreesclockwise or counterclockwise, the distance between the two tabletcomputers remains constant at four inches. In this example, a positionof the real object did not change since it remains unmoved on the table.

Still continuing with this example, while both the virtual and realtablet computers are in the field of view of the wearable electronicglasses, the real tablet computer moves ten inches farther away from alocation of the virtual tablet computer. During movement of the realtablet computer, a location of the virtual tablet computer on the lenschanges to maintain a consistent distance of four inches between thevirtual and real tablet computers. The virtual tablet computer moves tofollow a path or trajectory of the real computer such that a distance orseparation between the two objects appears to the wearer to beunchanged.

One example embodiment moves a position of the virtual object on adisplay of the wearable electronic device in order to maintain aconsistent position between the virtual object and the real object. Thismovement of the virtual object offsets or counters the movement of thewearable electronic device and/or the real object so the virtual objectappears to not move with respect to the real object seen through thewearable electronic device.

Consider an example in which a virtual object (e.g., a mobile phone) isdisplayed on a display or lens of a pair of wearable electronic glassessuch that this virtual object overlaps a real object (e.g., a mobilephone) that is in a field of view of a user wearing the glasses. Thevirtual object and the real object appear centered in the field of viewof the user, and the virtual mobile phone is a same make, model, andsize of the real mobile phone and is overlaid on top of the real mobilephone. The user rotates his head forty-five degrees counterclockwise. Ifa position of the virtual mobile phone did change with respect to thedisplay or lens, then the virtual mobile would no longer appear overlaidupon the real mobile phone but appear forty-five degrees in acounterclockwise position removed from the real mobile phone. In orderto counteract this movement, a position of the virtual mobile phone onthe display or lens of the wearable electronic glasses is changed. Inthis example, the virtual mobile phone is rotated forty-five degreesclockwise to counter the forty-five degree counterclockwise movement ofthe wearable electronic glasses. Thus, the virtual mobile phone moves inan opposite direction to that of the wearable electronic glasses inorder to generate an appearance that the virtual mobile phone did notmove with respect to the real mobile phone.

In an example embodiment, movement of the virtual object on a lens ordisplay of the wearable electronic device is equal to and opposite tomovement of the wearable electronic device and/or the real object. Thisequal and opposite movement of the virtual object occurs simultaneouslyand in real-time with the movement of the wearable electronic glassesand/or the real object. In this manner, the virtual object appears notto move to a user and thus remains in a consistent relationship with thereal object while both objects are being viewed with the wearableelectronic glasses.

Consider an example in which the real object is a remote controlledmotorized vehicle that a user views with a wearable electronic device. Avirtual image of this motorized vehicle appears to the user to be belowthe motorized vehicle. In response to receiving transmission signals,the motorized vehicle moves in a curved path with a northward direction.The virtual image follows the motorized vehicle and also moves in thecurved path with the northward direction while maintaining a consistentdistance from the motorized vehicle. During movement of the motorizedvehicle, neither the user nor the wearable electronic device moves. Ifthe wearable electronic device also moved during movement of themotorized vehicle, then the virtual image would move to offset and/orcounter this movement as well.

Consider an example in which a user wears a pair of wearable electronicglasses that projects an image of a soda can (i.e., the virtual object)next to a real glass with ice (i.e., the real object). While viewing thesoda can and the glass, the user moves his head ten degrees upward andtwenty degrees clockwise. In order to counter this movement, thewearable electronic glasses changes a position of the projection of thesoda can ten degrees downward and twenty degrees counterclockwise. Froma point of view of the user, the soda can and the glass did not move.This change of the projection of the soda can causes the soda can toappear to remain unchanged or unmoved next to the glass.

Consider an example in which a user wears a pair of wearable electronicglasses that provides a virtual image of a document with written text(i.e., the virtual object) on top of a flat, white surface (i.e., thereal object). The virtual document with the written text appears toexist on the surface. While reading the text of the virtual document,the user looks away from the surface and simultaneously moves his headin the direction of the look. In order to counteract this head movement,the wearable electronic glasses shifts or moves the virtual text with aspeed and direction that are equal and opposite the speed and directionof the movement of the head of the user. This counteraction causes thevirtual document to appear to remain still and unchanged while beingprovided on the surface.

FIG. 6 is a method to move a location of a virtual object in response tomovement of a wearable electronic device and/or a real object.

Block 600 states display, through and/or with a wearable electronicdevice, a virtual object with a real object.

The virtual object can appear next to, above, below, besides, on, over,near, superimposed on, overlaid with, or partially on the real object.For instance, while the real object is visible with or in the field ofview through the wearable electronic device, the virtual object alsoappears in the field of view. Thus the virtual object and the realobject simultaneously or concurrently appear in the field of view of thewearable electronic device.

Block 610 states move, in response to movement of the wearableelectronic device and/or the real object, the virtual object from beingat one location with respect to the real object to being at anotherlocation with respect to the real object.

The virtual object moves around the real object as the distance betweenthe virtual object and a perimeter or edge of the field of view changes.Movement of the wearable electronic device and/or the real object cancause a space between the virtual object and the perimeter to decreasesuch that there is no longer sufficient space for the virtual object atits current location in the field of view of the wearable electronicdevice. When the space is no longer sufficient, the virtual object movesto a new location in the field of view with respect to the real object.This action of moving the virtual object around the real object orwithin the field of view maintains the virtual object in the field ofview while the wearable electronic device and/or the real object moves.This action also extends an amount of time that the virtual objectappears with the real object. If the virtual object were not moved, thenit would disappear from the field of view of the wearable electronicdevice and hence not be visible to the wearer.

For example, as the wearable electronic device moves, a distance orspace between an edge or perimeter of the field of view and the realobject and/or the virtual object can increase or decrease depending onthe direction of movement of the wearable electronic device. Forexample, if the real object is located in a center of the field of viewof the wearable electronic device and a head of a user moves to hisleft, then space from a right side of the perimeter of the field of viewdecreases while space from a left side of the perimeter of the field ofview simultaneously increases. If the virtual object were displayed onthe right side of the real object, then movement of the wearableelectronic device to the left may cause the virtual object to falloutside of the field of view. In other words, as the wearable electronicdevice continues to move to the left, a distance decreases between thevirtual object and a right side of the perimeter of the field of view.If the virtual object does not move its location, then the virtualobject will be pushed out of the field of view of the wearableelectronic device. To compensate for this movement, the virtual objectis moved to a location in the field of view where more space existsbetween the real object and the perimeter. In this example, the virtualobject could be moved from a right side of the real object and to a leftside of the real object.

The virtual object can switch or move locations with respect to the realobject as the wearable electronic device moves and alters the field ofview for a user and/or as the real object moves within the field ofview. When an available space in the field of view for the virtualobject shrinks or becomes too small to house or include the virtualobject, the virtual object moves or switches to a new location that hasmore space in the field of view. Thus, the virtual object can switch ormove locations within the field of view and with respect to the realobject in order to compensate for movement of the wearable electronicdevice and/or movement of the real object.

By way of example, the virtual object moves to a new location in thefield of view when its current location becomes too small to accommodatea size and/or shape of the virtual object. This new location changes aposition of the virtual object with respect to the real object. Forexample, the virtual object moves from one side of the real object to anopposite side of the real object (such as moving from above the realobject to below the real object or moving from a right side of the realobject to a left side of the real object or moving from being in frontof the real object to being behind the real object).

Consider an example in which a user wears a pair of wearable electronicglasses that provides an elliptical field of view. A real object islocated in a center of the ellipse, and a virtual object is displayed onthe lens of the glasses and next to and to the left of the real objectfrom the point of view of the user. As the user moves his head to hisright, an edge or a perimeter on a left side of the field of viewsimultaneously moves toward the virtual object. This movement shrinks anavailable space in the field of view for the virtual object. If the usercontinues to move his head in the rightward direction, then the virtualobject will be outside of the field of view of the user and hence nolonger visible. To prevent this from happening, the virtual object movesto a new location with respect to the real object. In this instance,since the user is moving his head to the right, available space on aright side of the real object increases. The virtual object switchesfrom being located at a left side of the real object to being located atan opposite right side of the real object to counteract the movement ofthe wearable electronic device and change in the field of view.

Consider an example in which a user wears a pair of wearable electronicglasses that provides a field of view that includes a computer (i.e.,the real object) and a virtual computer (i.e., the virtual object). Thevirtual computer is situated to a right side of the real computer. Aperson picks up the real computer and moves it to the location that thevirtual computer occupies. In response to this movement, the virtualcomputer changes location and moves to a location previously occupied bythe real computer (i.e., move the virtual computer to the location wherethe real computer was located right before the person moved it).

Consider an example in which the virtual object is located in front ofthe real object from the point of view of the user (i.e., the virtualobject is located between the user and the real object). The user movestoward the virtual object and the real object such that the user nowphysically occupies the space in the real world where the virtual objectappeared to be. In response to this movement, the virtual object ismoved to a new location that is located behind the real object (i.e.,now the real object is located between the user and the virtual object).

Consider an example in which the wearable electronic device presents auser with a square or rectangular shaped field of view with a realobject located in a center of the field of view and a virtual objectlocated adjacent to the real object. As the wearable electronic devicemoves, the real object is no longer located in a center of the field ofview. In response to this movement, the virtual object changes orswitches locations with respect to the real object. For instance, thevirtual object moves from one corner of the square or rectangular fieldof view to another corner of the field of view. The virtual object canalso move from being located above the real object to being locatedbelow the real object or from being located on one side of the realobject to being located on an opposite side of the real object.

The virtual object can move to various locations within the field ofview in order to extend a length of time in the field of view as thewearable electronic device and/or the real object moves. For example, asthe wearable electronic device and/or real object moves, the virtualobject moves in order to increase a length of time that the virtualobject is visible within the field of view.

Furthermore, the virtual object can move to various locations within thefield of view in order to avoid colliding with an edge of the field ofview. For example, as the wearable electronic device moves, the virtualobject moves to different locations around a perimeter of the realobject to avoid occupying a location that would be outside of the fieldof view.

Furthermore, the virtual object can move to locations in the field ofview with a maximum size or maximum available or free space toaccommodate the virtual object. For instance, if a greatest or largestunoccupied area in the field of view is located below the real object,then the virtual object moves to occupy this area.

The virtual object can change position, orientation, shape, and/or sizewith respect to the real object in response to movement of the realobject and/or wearable electronic device (even if the wearer of thewearable electronic device of the wearable electronic device itself doesnot move). For example, as the real object moves within the field ofview of the wearable electronic device, the virtual object also moves inthis field of view in response to these movements.

As one example, when a user picks up the real object and moves it onefoot to his right, then the virtual object simultaneously moves one footto the right. When the real object moves, the virtual object also movesto maintain a consistent relationship with respect to the real object.The virtual object can track or follow movements of the real object.

As another example, a virtual object is shown with a perspective view toa user in a field of view of electronic glasses. A real object also inthis field of view moves toward the virtual object. In response to thismovement, the electronic glasses change the view and present the virtualobject with a plan view to the user.

As another example, a virtual object is shown rotating as rotating aboutan imaginary x-axis in the real-world to a user in a field of view ofelectronic glasses. A real object also in this field of view movestoward the virtual object. In response to this movement, the electronicglasses change the virtual object from rotating about the x-axis torotating about a z-axis.

As another example, a user wears a pair of wearable electronic glassesand views a person that is the real object. A virtual image of thisperson appears as the virtual object next to the real person. As theperson (i.e., the real object) walks away from the user, the virtualimage of the person (i.e., the virtual object) simultaneously walks awayfrom the person. The virtual object emulates the walking movement of theperson such that the arms, legs, body, head, etc. of both the real andvirtual persons move in stride or in synchronization with each other asthe real person walks away. Additionally, as the real person walks away,he becomes smaller (i.e., as the person moves farther away from theuser, the person appears smaller to the user because the visual angle ofthe user decreases). A size of the virtual object contemporaneouslydecreases such that the two objects appear to get smaller together asthe person walks away into the distance.

FIG. 7 is a method to display, with and/or through a wearable electronicdevice, actions to virtual objects received from an electronic device.

Block 700 states provide or capture, with a wearable electronic device,a real location with real objects.

For instance, a user wears the wearable electronic device that offers afield of view through its lens of the real location in the real-worldand the real objects located in the real location. This real locationcan be a place (such as a third floor of a building), a scene (such as aview from a window), a geographical location (such a Global PositioningSystem, GPS, location on earth), an area (such an interior of a retailstore), etc.

Block 710 states display, on a display of an electronic device, avirtual location with virtual objects that emulate the real location andthe real objects.

The virtual location emulates, copies, or represents the real location,and the virtual objects emulate, copy, or represent the real objects.The virtual location and the virtual objects have one or more featuresthat match or imitate the respective real location and real objects. Forinstance, a display of an electronic device displays a virtual hospitaloperating room that emulates a real hospital operating room that isremotely located from the electronic device. The virtual hospital roomthus appears to be the real hospital room.

The electronic device can receive or obtain the real location and/orreal objects from memory, a transmission, the wearable electronicdevice, another electronic device, etc.

Block 720 states receive, at the electronic device, actions to one ormore of the virtual objects at the virtual location.

An action (such as a thing done, a movement, an accomplishment, or atask) is performed on or with respect to one or more of the virtualobjects being displayed in the virtual location. For instance, a userinteracts with a user interface (UI) or graphical user interface (GUI)of the electronic device to select, move, highlight, point to, grab,alter, or remove one of the virtual objects. As another example, a usercovers or blocks a virtual object in the virtual location with his hand,his arm, or a person that appears in the virtual location. As yetanother example, a person at the electronic device interacts with anaugmented reality system to impose an image at the virtual locationand/or with the virtual objects.

Block 730 states display, through and/or with a display of the wearableelectronic device, the actions such that the actions to the one or morevirtual objects at the virtual location are reproduced with respect tothe real objects at the real location.

The actions to the one or more virtual objects and/or at the virtuallocation are reproduced, displayed, emulated, copied, replicated, and/orprovided to the corresponding one or more real objects and/or reallocation at the wearable electronic device.

Consider an example in which a handheld portable electronic device(HPED) displays an interior of a virtual retail store that sellsgroceries. This virtual retail store emulates or looks like a realretail store in the real-world that includes aisles with shelves of fooditems that are available for selection and purchase. The virtual retailstore being displayed on the HPED also includes the same aisles,shelves, food, etc. as the real retail store. A user interacts with aGUI of the HPED to select a can of soup on a third shelf of aisle fivein the virtual store. The user drags this can of soup from the shelfinto a shopping cart. A wearer of a wearable electronic devicephysically enters the retail store and proceeds to the aisle five. Inthis aisle, the third shelf has cans of soup. One of the cans of soup isvirtually marked or distinguished to instruct the wearer that the userselected this particular can of soup. For example, the can of soup ishighlighted or outlined with a color or other indicia on the display ofthe wearable electronic device so the wearer is able to determine andselect the correct can of soup. The wearer grabs the physical can ofsoup from the third shelf in aisle five and places the can of soup in ashopping cart. The selected can of soup is then delivered to the home ofthe user of the HPED.

Consider an example in which a factory warehouse stores large numbers ofcrates that are moved on a daily basis and are shipped to differentparts of the world. A foreman of the warehouse uses a tablet computer todisplay a virtual or real image of the warehouse. The foreman selectscrates from one location and moves them to another location. Next, theforeman opens, on the display, a shipping container, selects items fromthe container, and moves them to different locations through thewarehouse. These actions of the foreman with the warehouse are stored ona cloud server and are assigned as a workflow to a forklift operatornamed John. Later, John comes to work, dawns a pair of wearableelectronic glasses, and retrieves the workflow from the server. Theworkflow commences and John sees the crates (previously selected by theforeman) as being highlighted. John drives the forklift to this locationand moves the crates to the different locations that are indicated inthe workflow. After completion of these tasks, the wearable electronicglasses show the next series of tasks of moving the items from theshipping container. Actions to be performed on the physical items in thewarehouse are displayed on the wearable electronic glasses such thatJohn knows which items to select, where to move the items with theforklift, and a sequence of the workflow per the instructions of theforeman on the tablet computer.

Consider an example in which an elderly man desires to have a clerk gogrocery shopping on behalf of the man. The clerk wears a pair ofwearable electronic glasses, and video from these glasses appears on adisplay of an electronic device of the man. While the clerk is at thestore, the man interacts with the display to instruct the clerk where togo in the store. When the man sees on his display a food item that hedesires, he taps the display to highlight the food item. This actionsimultaneously causes the clerk to see a virtual identification throughthe lens or with the display of the wearable electronic glasses. Thus,actions of the man with his display appear in real-time as virtualidentifications or visualizations with the wearable electronic glasses.For example, the real food item and its location in the store are markedor distinguished with the wearable electronic glasses so the clerk knowswhich item to select for the man. In this manner, the clerk acts as aproxy or agent for the man and performs the grocery shopping per thereal-time instructions from the man.

FIG. 8 is a method to display a translation of a writing on a realobject.

Block 800 states detect or provide, with a wearable electronic device, areal object with writing in a first language.

The wearable electronic device detects or provides an object or adocument with writing in the field of view of the wearable electronicdevice. A determination is made of the language of the writing. Forexample, a paper document on a desk of a user is written in Frenchlanguage. The user wears a wearable electronic device while viewing thedocument, and the wearable electronic device communicates wirelesslyover a cloud with an optical character recognition (OCR) engine thatdetermines the document is written in French. As another example,writing on a computer screen is written in Chinese and viewed with auser that wears a wearable electronic device.

Block 810 states translate the writing from the first language into asecond language that is different than the first language.

The wearable electronic device and/or another electronic devicetranslate the writing of the real object into another language. Forexample, if a document is written in Chinese, then the wearableelectronic device translates the writing on the document into English. Auser or electronic device can select the language into which the writingon the document is translated. As another example, a real objectincludes writing in Spanish. This writing is translated with a serverfrom Spanish to English, and the translation is transmitted to thewearable electronic device.

Block 820 states display, through and/or with the wearable electronicdevice, the writing in the second language over or on the real objectsuch that the writing on the real object appears to be written in thesecond language.

Writing on the real object is translated into another language, and thistranslated writing is superimposed on or provided with the real objectwith the wearable electronic device. To a wearer of the wearableelectronic device, the real object appears to be written in the secondlanguage. For example, a virtual image of the writing in the secondlanguage appears over and covers the writing in the first language.

Consider an example in which a user navigates with an HPED to a websitethat is written in English. The user wears a pair of wearable electronicdevice that stores or communicates with optical character recognition(OCR) software or engine that translates the written English text of thewebsite into Spanish. This Spanish text is then displayed through thewearable electronic device such that the website appears to be writtenin Spanish while the user views the website through the wearableelectronic device. Thus, even though the website is actually written inEnglish, the website appears to the wearer of the wearable electronicdevice to be written in Spanish. Virtual Spanish text is displayed on orthrough the lens of the wearable electronic device in locations thatcorrespond with the respective English text on the website.

Consider an example in which a user types a letter in English on hisnotebook computer. After the letter is written, the user dawns awearable electronic device, selects French language, and views letterwith the wearable electronic device. The English text of the letter istranslated into French that appears through a display of the wearableelectronic device. Thus, as the user views the letter on his notebookcomputer with the wearable electronic device, the letter appears to bewritten in French.

Consider an example in which a user living in Canada receives a box inthe mail from China. An exterior surface of the box includes writing inChinese. The user is unable to read Chinese and views the box with awearable electronic device that includes a display. This display showsthe Chinese writing to be in English. Thus, from the point of view ofthe user, the exterior surface of the box appears to be written inEnglish, not Chinese. Virtual text in English appears on the box at thelocations with Chinese text.

FIG. 9 is a method to display virtual content associated with an area oran object when a wearable electronic device views the area, views theobject, and/or comes within proximity of the area and/or the object.

Block 900 states store virtual content that is associated with an areaand/or an object.

Virtual content is stored in memory, such as being stored on a cloudserver, a wearable electronic device, or an electronic device. Thisvirtual content is associated with an area (such as a geographical areaor location) and/or an object (such as a person, a thing, or somethingthat is visible or tangible).

Block 910 states display the virtual content when a wearable electronicdevice views the area, views the object, and/or comes within proximityof the area and/or the object.

Virtual content is retrieved, transmitted, obtained, displayed, oractivated when the wearable electronic device views the area, views theobject, and/or comes within proximity of the area and/or object.

Consider an example in which a user purchases a new computer thatincludes virtual content (such as virtual operating instructions). Theseinstructions play through a wearable electronic device as virtual imagesor virtual video. For instance, upon purchasing the computer, the useris granted access to 3D virtual instructions on using and operating thecomputer. The user dons a pair of wearable electronic glasses and viewsthe computer with the glasses to activate the instructions. While thecomputer is located within the field of view of the glasses, virtualinstructions play through the glasses. The user sees the instructionsand hears audio associated with the instructions simultaneously whileviewing the computer. These instructions assist the user in performingactions on the computer (such as assisting the user to start or operatethe computer, connect to a network or other electronic device, downloador upload data, configure settings, etc.). These instructions can playnext to, over, or near the computer that is within the field of view ofthe glasses. The instructions stop playing when the user looks away fromthe computer such that the computer is no longer in the field of view ofthe user.

Consider an example in which a person (John) records a video of himselftalking in order to leave a virtual video message for his friend(Peter). The video is transformed into a 3D virtual image of John withhis voice as audio. John transmits the video recording to Peter withplayback instructions to replay the video recording when Peter entershis office. Peter receives a message on his HPED that indicates he has amessage from John, but Peter cannot retrieve or view the message untilhe is located at the playback location. When Peter arrives at hisoffice, he retrieves and activates the video recording with his wearableelectronic device (e.g., an electronic wristwatch with a display). Peterviews the virtual video recording of John while sitting in his office.Activation, retrieval, and/or viewing of the message were contingent onPeter being at a certain or predetermined geographical location (i.e.,his office). The message would not play until Peter was physicallylocated at the office since Peter was required to be present at thislocation to retrieve and/or activate the message from memory.

Consider an example in which a mother bakes cookies and places them on aplate on a counter in the kitchen. While wearing a pair of electronicglasses, she views the plate of cookies and activates a message mode.The electronic glasses sense her hand that draws a virtual red circle inthe air around the cookies and records her voice saying, “Do not eatuntil after dinner.” Later, the mother's son enters the kitchen whilewearing his electronic glasses. His field of view encounters the plateof cookies, and he sees a virtual red circle around the plate ofcookies. He enters message mode with his glasses, points his hand towardthe cookies, and a verbal message is played from this glasses saying, inhis mother's voice, “Do not eat until after dinner.” Thus, playback ofthe message and viewing of the virtual content were tied to the sonbeing at the physical location and looking in a particular direction(i.e., having the plate of cookies being in his field of view with theglasses).

Consider an example in which a person (Sally) interacts with a tabletcomputer to shop at an online retail store (named Store ABC). Anavigable image of the store appears on the tablet computer as a virtualstore. This virtual store has a physical, real store as a counterpart ina shopping complex (i.e., Store ABC exists in a shopping complex and asa virtual store accessible over the Internet). Thus, a location andexistence of items in the virtual store shown on the tablet computercorrespond with a location an existence of the same items in the actualretail store. Sally interacts with the tablet computer to select a blackshirt with size medium, which is the last black shirt available in thissize. When she selects the shirt with the tablet computer, the shirt ismarked or tagged in both the retail store and the virtual online storeas being sold, taken, or not available. Sally finishes shopping and paysfor the shirt. The actual or physical shirt remains in the retail storesince the shirt has not yet been retrieved by Sally or by a storeemployee. Later, while wearing a pair of electronic glasses, anotherperson (Mary) goes to the shopping complex and enters Store ABC. Maryintends to buy the same black shirt with size medium as Sally previouslypurchased. Mary finds the black shirt but notices a virtual indicationor mark on the black shirt that notifies her that the shirt is sold andnot available to purchase. This indication or mark appears as a virtualimage on or through the display of her electronic glasses.

Consider an example in which a user wears a wearable electronic devicewhile shopping in a store. When the user comes within ten feet ofclothes rack, a 3D virtual advertisement for the clothes on the rackbegins to play through the wearable electronic device of the user. Thisadvertisement automatically plays on wearable electronic devices ofother users when these users physically enter within a radius of tenfeet from the clothes rack.

Virtual content associated with an area and/or an object can also bedisplayed when a user focuses her or her eyes on the area and/or object.Focus of the eyes on the area and/or object triggers or activates thevirtual content. For example, the wearable electronic device includes,uses, or communicates with eye tracking software to determine adirection of gaze or focus and/or where the eyes of the user arefocused.

Consider an example in which a wearer of a wearable electronic devicefocuses on a real object with his eyes. When the wearer focuses on thisobject, a virtual object that emulates this real object activates andappears on a lens or a display of the wearable electronic device. Whenthe wearer looks away and changes his eyes from focusing on this object,then the virtual object disappears and/or is removed from appearing onthe lens or the display.

Consider an example in which a user wears electronic glasses and shopsin a retail store with various racks and items. A field of view of theuser includes three different clothes racks. When the user focuses hereyes on one of the racks, an advertisement for clothes in this rackappears and plays with the electronic glasses. Advertisements for theother two clothes racks do not play for the user since her eyes are notfocused on those racks. The electronic glasses determine on which rackher eyes are focused and retrieve and play the advertisements for thatrack.

Consider an example in which a user wears a wearable electronic devicethat includes an eye tracker that measures or determines a point of gazeof the user and/or motion of an eye relative to a position of the head.The user walks in an airport that includes various physical signs. Whenthe user focuses on one of these signs, additional information orinstructions are visually displayed to the user through or with thewearable electronic device.

FIG. 10 is a method to provide instructions to a wearable electronicdevice for how to cure an error with an electronic device.

Block 1000 states determine an error or problem with an electronicdevice.

The electronic device can diagnose its own error or problem or receivethis diagnosis from another electronic device. For example, a printerdetermines that it has a paper jam in tray number two. As anotherexample, a tablet computer determines that it has a virus or a corruptfile. As another example, a server in communication with an HPEDdetermines that the HPED has outdated software and notifies the HPED ofthis determination. As another example, a wearable electronic devicecommunicates with the printer, and the two electronic devices determinethat the printer is offline and not connected to the Internet.

Block 1010 states retrieve, receive, or provide instructions for how toremedy or cure the error or problem.

For example, the electronic device retrieves the instructions from itsown memory or from another memory (such as receiving the instructionsfrom a cloud computing device that is in communication with theelectronic device). The instructions can include video, audio, and/orvirtual images that instruct how to repair, fix, cure, remedy, or solvethe error or problem.

Block 1020 states provide the instructions to a wearable electronicdevice.

By way of example, the electronic device transmits the instructions tothe wearable electronic device or provides the wearable electronicdevice with a storage location or access for retrieving the instructionsfrom another electronic device (such as providing the wearableelectronic device with information to enable the retrieval of theinstructions from a cloud server). Alternatively, the instructions areretrieved from memory of the wearable electronic device itself. Forinstance, a wearable electronic device determines that it has a problem,retrieves instructions to cure this problem, and stores the instructionsin memory.

Block 1030 states provide, by the wearable electronic device, theinstructions to one or more persons.

For example, the wearable electronic device displays, portrays,transmits, broadcasts, or projects the instructions such that theinstructions can be viewed or heard by a wearer or user of the wearableelectronic device.

Consider an example in which a printer determines that its ink cartridgeis low and needs replaced. The printer sends a user a message to replacethe ink cartridge. The user purchases a new ink cartridge but does notknow how to replace the old ink cartridge with the new ink cartridge.The printer sends a sequence of virtual 3D video instructions to theuser while the user is wearing a pair of electronic glasses. Theseinstructions overlay on or superimpose on the printer and show the userstep-by-step instructions on how to replace the old ink cartridge withthe new ink cartridge. For example, step one shows a virtual front coverof the actual printer opening while the user views the actual printer.After the user opens the front cover of the printer, step two plays.This process continues until the user successfully changes the inkcartridge.

Consider an example in which an automobile determines that its engineoil is low and provides a message to the driver to add oil. The driver,however, does not know how to add oil to the automobile and requestsvirtual assistance. In response to this request, the automobiletransmits instructions how to add oil to a wearable electronic device ofthe user. These instructions play a 3D video in augmented reality thatshows step-by-step instructions on how to add oil to the automobile. Thevideo plays while the user actually performs the task of adding oil.After the user completes one of the steps, then the video proceeds tothe next step. For instance, the video pauses or stops after each step,waits for a confirmation that this step was successfully completed,plays the next step, pauses, waits for a confirmation that this step wassuccessfully completed, etc.

FIG. 11 is a method to share a location of a virtual object withmultiple wearable electronic devices.

Block 1100 states generate, on or to a real object or a real area, avirtual object that is visible to a user with a wearable electronicdevice.

For example, a virtual object is generated to, provided to, or retrievedto an object or an area that exist in the real world. This virtualobject is visible with or through a wearable electronic device.

Block 1110 states store a location of the virtual object with respect tothe real object and/or the real area.

The location of the virtual object is stored in memory. This locationcan include a mechanism to establish the location with respect to thereal object or the real area, such as a tag, a coordinate, a marker, avisual reference, a GPS location, feature detection (such as cornerdetection, blob detection, edge detection, color detection), mapping,image processing, etc. By way of further example, augmented reality usesimage registration to determine and restore real world coordinates.

Block 1120 states share the location of the virtual object with multiplewearable electronic devices such that the virtual object is visible toother users with the multiple wearable electronic devices.

For example, an electronic device transmits the location to the multiplewearable electronic devices. Users of these devices can simultaneouslyview the virtual image or virtual object at the location from differentpoints of view or from different physical locations.

Consider an example in which different users are located in a room andwear wearable electronic devices. One of these users generates a 3Dimage that appears in the middle of the room. The wearable electronicdevice of this user transmits this image and its location to a server,and the server transmits the image and location to the other wearableelectronic devices in the room. Each user can now see the 3D image thatthe user generated.

Consider an example in which users play a war game or a battle game.Each player carries a pretend gun that fires virtual ammunition (such asvirtual bullets, lasers, projectiles, etc.) and wears a pair ofelectronic glasses. An object of the game is to shoot other players withthe pretend gun to score points. A first player uses his pretend gun toshoot virtual shots at a second player. These virtual shots miss thesecond player but hit a wall and thus leave virtual bullet holes in thewall. A location of these shots, a number of shots, and damage to thewall are recorded and stored. Subsequently, when the first player,second player, or other players look at the wall, they see the wall withthe bullet holes. The wall thus appears to all of the players to havebeen shot with bullets. During the game, the second player returns fireat the first player and a virtual bullet from the second player grazesan arm of the first player. Visual affects of the game attempt to mimicreality. In reality, this bullet would have caused a wound in the arm ofthe first user, so this wound is imitated in the virtual sense.Specifically, the first user appears to have a wound on his arm, thoughthe wound is virtual. When the first player or other players view thearm of the first player, they see the wound caused from the virtualbullet. This wound remains with the first player as he moves around todifferent physical locations and continues the game.

Consider an example in which two people are standing next to each otherand are wearing electronic glasses. Both people see a virtual printerthat is located in front of them on a real table (i.e., a virtualprinter is located on top of a real table). One of the wearers extendshis hand to the virtual printer and raises a lid to the printer.Movements of the hand of the user are detected to determine the actionof raising the lid of the virtual printer. In response to thesemovements, the lid of the virtual printer raises. Both people see thevirtual printer with the raised lid.

FIGS. 12A-12D show a computer system or electronic device system 1200that provides instructions to a wearable electronic device. This system1200 includes a wearable electronic device 1210 worn on a user 1220 andan electronic device 1230 (such as a HPED) that communicates with thewearable electronic device 1210 over a network 1240.

A field of view 1250 is visible through or with the wearable electronicdevice 1210. By way of illustration, this field of view 1250 has anelliptical shape to represent a view or field of focus of the user 1220through electronic glasses. Dashed lines 1260 represent a perimeter,periphery, or edge of the field of view 1250.

FIG. 12A shows the user 1220 looking straight ahead with a field of viewthat includes an automobile 1262 and a tree 1264 near the automobile.The wearable electronic device 1210 captures this image and transmits itthrough the network 1240 to a display 1270 of the HPED 1230. As such, aview of the wearable electronic device appears on the display of theHPED.

The display 1270 of the HPED 1230 includes an automobile 1272 and a tree1274 near the automobile that represent the automobile 1262 and tree1264 in the field of view 1250 for the wearable electronic device 1210.Real objects in the field of view 1250 can appear as similar oridentical objects on the display 1270 of the HPED 1230. For instance,the real objects of the automobile 1262 and the tree 1264 appear as realobjects of the automobile 1272 and the tree 1274. By way of example, thewearable electronic device 1210 captures video of the real objects, andthis video is reproduced on the display 1270 of the HPED 1230. Realobjects in the field of view 1250 can also appear as altered objects onthe display 1270 of the HPED 1230. For instance, the real objects of theautomobile 1262 and the tree 1264 are transformed and appear as virtualobjects (e.g., the automobile 1272 and the tree 1274 are virtual imagesof the real automobile and the real tree). Further yet, the real objectsin the field of view 1250 of the wearable electronic device 1210 can bealtered. For instance, a virtual image of an automobile is included withor superimposed over the automobile 1262, and virtual image of a tree isincluded with or superimposed over the tree 1264. As another example,the automobile 1262 is replaced with a virtual automobile that emulatesthe real automobile, and the tree 1264 is replaced with a virtual treethat emulates the real tree.

FIG. 12B shows instructions being issued from the HPED 1230, overnetwork 1240, and to the wearable electronic device 1210. Forillustration, the tree 1274 displayed on the HPED 1230 is marked,highlighted, or otherwise distinguished from other objects beingdisplayed on the display 1270. For example, a virtual image orindication 1280 is superimposed on or provided with the tree 1274 on theHPED to visually illustrate or distinguish to a user the tree from theautomobile 1272.

The indication 1280 (shown on the tree 1274 of the HPED) replicates orcopies to the tree 1264 in the field of view 1250 of the user 1220. Theuser 1220 sees a virtual image or indication 1282 that distinguishes thetree 1264 from other objects (such as the automobile 1262) in the fieldof view 1250.

For illustration, the indications are shown as virtual parallel linesthat appear on the tree 1274 on the HPED 1230 and on the tree 1264 inthe field of view 1250. Other indications can be used, such as usinghighlights, colors, shading, non-parallel lines, light, etc. to mark ordistinguish the tree. By way of example, a user interacts with the HPEDand uses a user interface (UI) or graphical user interface (GUI) to addvirtual marks to distinguish or to emphasize the tree. Furthermore, theindications shown on the HPED can be the same or different than theindications shown in the field of view.

Consider an example in which a user interacts with the HPED 1230 inorder to control or instruct the user 1220 that wears the wearableelectronic device 1210. The user of the HPED desires to instruct theuser of the wearable electronic device to walk to the tree 1264. Inorder to provide this instruction, the user marks the tree 1274 and thismark appears as a virtual mark, virtual indicia, or virtual indicationon the tree 1264. While wearing the wearable electronic device 1210, theuser 1220 sees a virtual indication 1282 on or at the tree 1264 andproceeds in that direction as instructed.

FIG. 12C illustrates the system 1200 in which voice control oractivation assists in providing instructions from the HPED 1230, overnetwork 1240, and to the wearable electronic device 1210. The display1270 of the HPED 1230 includes a visual indication (shown as “voice on”)to illustrate that the HPED is in voice control mode. Likewise, thefield of view 1250 of the wearable electronic device 1210 includes avisual indication (shown as “voice on”) to illustrate that the wearableelectronic device 1210 is in voice control mode. This mode enables usersto perform actions such as generating virtual instructions,communicating with each other using voice and issuing instructions orcommands to an electronic device with voice.

Consider an example in which a user interacts with the HPED 1230 inorder to control or instruct the user 1220 that wears the wearableelectronic device 1210. The user of the HPED desires to instruct theuser of the wearable electronic device to walk to the tree 1264. Inorder to provide this instruction, the user of the HPED speaks thefollowing instructions: “Walk to the tree.” In response to these verbalinstructions, the HPED 1230 recognizes a tree in the view, marks thetree, and transmits this instruction to the wearable electronic device.The HPED marks the tree with a virtual mark, virtual indicia, or virtualindication on the tree 1264 and on the tree 1274. The user of the HPEDsees the virtual indication 1280 on the tree 1274 and can confirm thatthe HPED marked or selected the correct tree. While wearing the wearableelectronic device 1210, the user 1220 sees a virtual indication 1282 onor at the tree 1264 and proceeds in that direction as instructed. Inthis example, verbal commands are used to generate a virtual indication,apply this virtual indication to a real object, and transmit the virtualindication for the real object to another electronic device.

FIG. 12D illustrates the system 1200 in which the display 1270 of theHPED 1230 includes controls 1286 that provide instructions over thenetwork 1240 to the wearable electronic device 1210 that projects ordisplays controls 1288 in the field of view 1250. The controls 1288provided on the wearable electronic device 1210 can be similar to ordifferent than the controls 1286 provided on the HPED 1230. Forillustration purposes, the controls are shown as being similar on theHPED and the wearable electronic device.

By way of example, the controls 1286 include a head motion or headmovement control 1290, a speed control 1292, and a direction control1294. These controls are also shown as virtual objects or images in thefield of view 1250 of the wearable electronic device 1210. The headmovement control 1290 has a circular or doughnut shape with differentsections. When a section is highlighted or activated, the user will movehis head in the direction of this section. An activated section providesa user with instruction on which direction to turn or face his head. Thespeed control 1292 has a rectangular shape with different sections. Whena section is highlighted or activated, the user can determine a speed atwhich to move. For instance, activation of the lower section indicateswalk or move slowly; activation of half of the rectangle indicates walkquickly or jog slowly; and activation of the entire rectangle indicatesmove quickly or run. The direction control 1294 includes four arrows orpointers that point in different directions (such as pointing to zerodegrees, ninety degrees, one hundred and eighty degrees, and two hundredand seventy degrees). When an arrow or pointer is highlighted oractivated, the user will proceed or take action in this direction.

Consider an example in which a user of the HPED 1230 desires the user1220 of the wearable electronic device 1210 to walk to the automobile1262. The user of the HPED speaks a command or instruction (e.g., “Go tocar”), and this instruction appears in the field of view 1250 of thewearable electronic device 1210. Further instructions are also provided,such as activation of the top arrow of the direction control 1294 so theuser 1220 knows in which direction to proceed and activation of twolower sections of the speed control 1292 so the user knows to walkquickly toward the automobile. Once at the automobile, a left section ofthe head movement control 1290 activates so the user knows to turn hishead to the left in a direction indicated with the activated section.Activation of the head, speed, and direction controls can be with verbalcommands, gesticulations of a body, interaction with a GUI, etc.

Thus, the controls 1286 enable a user of one electronic device (e.g.,the HPED 1230) to control or provide movement instructions to a user ofanother electronic device (e.g., the wearable electronic device 1210).For illustration purposes, the controls 1286 on the display 1270 of theHPED 1230 also appear in the field of view 1250 of the wearableelectronic device 1210. These two control sets can be the same, similar,or different.

FIGS. 13A-13C show a wearable electronic device 1300 that provides avirtual object 1310 with a real object 1320. For illustration, a user1330 wears the wearable electronic device 1300 (shown by way of exampleas electronic glasses or electronic eyewear). The user 1330 has a fieldof view 1340 through the wearable electronic device 1300 with aperiphery, edge, or perimeter shown with dashed lines 1350.

The field of view 1340 of the user 1330 includes the real object 1320(shown by way of example as a notebook computer in a closed state oropen state) and the virtual object 1310 (shown as a virtual notebookcomputer in the closed state or open state). For illustration purposes,the virtual object 1310 (i.e., the virtual notebook computer) emulatesor copies the real object 1320 (i.e., the real notebook computer). Assuch, the virtual object can look like or be similar to the real object.For instance, the virtual object appears as a same make and modelnotebook computer as the real notebook computer. Alternatively, thevirtual object can be different than or dissimilar to the real object.

The virtual object 1310 is placed or positioned next to or adjacent tothe real object 1320. As such, the user simultaneously sees the realobject 1320 (which is physically and actually present in the real worldin front of the user) and the virtual object 1310 (which is notphysically present in front of the user but appears in front of the useras a virtual image or virtual object). The wearable electronic device1300 presents, provides, displays, generates, or projects the virtualobject 1310 to the field of view 1340 of the user 1330.

In FIGS. 13A-13C, the wearable electronic device 1300 uses the virtualobject 1310 to provide the user with instructions or commands withregard to the real object 1320. For example, the virtual object 1310moves to assist the user in operating the real object 1320.

Consider an example in which the user 1330 purchases a new notebookcomputer (shown as the real object 1320), but does not know how tooperate this electronic device. The user requests instructions orassistance from the wearable electronic device 1300. In response to thisrequest, the wearable electronic device 1300 produces a virtual image ofthe real notebook computer next to the real notebook computer. FIG. 13Ashows both the notebook computer of the real object 1320 and thenotebook computer of the virtual object 1310 in a closed state (i.e.,the lid of the notebook computer is closed). Next, the wearableelectronic device 1300 moves or opens the lid of the virtual notebookcomputer to illustrate this step or action for the user 1330 to take onthe real notebook computer. FIG. 13B shows the notebook computer of thereal object 1320 in the closed state with the notebook computer of thevirtual object 1310 in an open state (i.e., the lid of the notebookcomputer is open). In response to this visual instruction ordemonstration, the user 1330 emulates the action of the virtual notebookcomputer and opens the lid of the real notebook computer. The wearableelectronic device 1300 detects movement of the lid and opening of thiscomputer (i.e., detects the completion of the action of the user ofopening the real notebook computer). Next, the wearable electronicdevice 1300 provides a virtual hand 1360 that takes an action on thevirtual notebook computer (such as showing the user how to turn on oroperate the notebook computer). FIG. 13C shows the notebook computer ofthe real object 1320 in the open state with the notebook computer of thevirtual object 1310 receiving an action from the virtual hand 1360 of auser.

Example embodiments are not limited to the virtual image or virtualobject being near or adjacent the real object. The virtual object can beincluded with or on the real object as well.

FIG. 14 shows a wearable electronic device 1400 that provides a virtualobject 1410 located on a real object 1420. For illustration, a user 1430wears the wearable electronic device 1400 (shown by way of example aselectronic glasses or electronic eyewear). The user 1430 has a field ofview 1440 through the wearable electronic device 1400 with a periphery,edge, or perimeter shown with dashed lines 1450.

The virtual object 1410 is shown as a notebook computer with a lid in anopen position. This notebook computer is superimposed on or over thereal object 1420 that is a notebook computer with a lid in a closedposition. A base of the virtual notebook computer is over or on the baseof the real notebook computer and has a similar or same shape and sizeas the real notebook computer. Movement of the lid of the virtualnotebook computer from the closed position to the open position visuallyillustrates or instructs the user to move the lid of the real notebookcomputer from the closed position to the open position. Movement of thelid of the real notebook computer places this lid in a position thatcoincides with a position of the lid of the virtual notebook computer.For example, the wearable electronic device places the virtual notebookcomputer over the real notebook computer and opens the lid of thevirtual notebook computer. This action of opening the lid instructs theuser to then open the lid of the real computer.

In an example embodiment, the virtual object can emulate the real objectand have a same size and shape as the real object. For instance, if thereal object is an electronic device, then the virtual object is a samemake and model of electronic device. Alternatively, the virtual objectcan be different in size, shape, make, model, etc. from the real object.

In FIG. 14, the user 1430 raises the lid of the real notebook computerto coincide with a position of the lid of the virtual notebook computer.For instance, the user raises the lid until a position of the lidmatches a position of the virtual lid. When the positions match oralign, the user can visually see that the lid of the notebook computerwas raised to the correct location, and this step was successfully andcorrectly completed.

FIGS. 15A-15C show a computer system or electronic device system 1500that provides instructions to a wearable electronic device. This system1500 includes a wearable electronic device 1510 worn on a user 1520 andan electronic device 1530 (such as a HPED) that communicates with thewearable electronic device 1510 over a network 1540.

A field of view 1550 is visible through or with the wearable electronicdevice 1510. By way of illustration, this field of view 1550 has anelliptical shape to represent a view or field of focus of the user 1520through electronic glasses. Dashed lines 1560 represent a perimeter,periphery, or edge of the field of view 1550.

FIG. 15A shows the user 1520 looking straight ahead with a field of viewthat includes a real object 1570 shown by way of example as a notebookcomputer. The wearable electronic device 1510 captures this image andtransmits it through the network 1540 to a display 1575 of the HPED1530. As such, a view of the field of view 1550 appears to the user 1520through the wearable electronic device and also on the display 1575 ofthe HPED 1530. The display 1575 of the HPED 1530 includes this image asnotebook computer 1574.

The display 1575 and the field of view 1550 also include a virtualobject 1580 shown as a virtual image of the real notebook computer 1570.This virtual object 1580 can be manipulated, moved, or changed with theHPED 1530 to instruct actions with regard to the real object 1570. Forexample, a user of the HPED 1530 or program executing on or with theHPED instructs the user 1520 through the wearable electronic device1530.

FIG. 15B shows the virtual object 1580 with its lid in an open position.For example, a user of the HPED 1530 interacts with the display 1575 orHPED to transition the lid or cover of the virtual notebook computerfrom a closed position to an open position. Actions on the virtualnotebook computer at the HPED 1530 simultaneously and in real-timeemulate on or reproduce to the virtual notebook computer shown in thefield of view 1550. As the lid opens on the virtual notebook computer atthe HPED 1530, the lid also opens on the virtual notebook computer atthe wearable electronic device 1510. In response to this instructionshown on the virtual object 1580, the user 1520 raises and opens the lidof the real object 1570 (i.e., the real notebook computer).

FIG. 15C shows the real object 1570 with its lid in an open position.The user 1520 raises and opens the lid of the real notebook computer.Actions on the real notebook computer 1570 simultaneously and inreal-time emulate on or reproduce to the image of the real notebookcomputer 1574 on the HPED 1530.

FIGS. 15A-15C show an example embodiment in which two persons locatedremotely from each other interact with electronic devices to provideinstructions with how to operate, control, or use a real object.

Consider an example in which the user 1520 was unable to operate hisnotebook computer 1570 (or other type of object or electronic device).The user requests assistance from a technician who is remote from theuser 1520. The technician views an image 1574 of the notebook computer1570 and executes a virtual image 1580 to assist in instructing the user1520. This virtual image appears to both the technician and the user1520. The technician manipulates the virtual image 1580 appearing on theHPED 1530 in order to provide instructions to the user on how to operatethe real notebook computer 1570. Instructions from the technician appearas visualizations (such as text or indicia) or movements on or with thevirtual object 1580. When the user takes an action on the notebookcomputer 1570, these actions are reproduced on the image 1574 at theHPED so the technician can see that the correct actions were taken.Alternatively, the HPED 1530 and/or a software program confirms that theactions of the user on the real notebook computer 1570 coincide with theinstructions shown with the virtual object 1580.

FIGS. 16A-16C show a wearable electronic device 1600 that provides avirtual object 1610 with a real object 1620. For illustration, a user1630 wears the wearable electronic device 1600 (shown by way of exampleas electronic glasses or electronic eyewear). The user 1630 has a fieldof view 1640 through the wearable electronic device 1600 with aperiphery, edge, or perimeter shown with dashed lines 1650.

The field of view 1640 of the user 1630 includes the real object 1620(shown by way of example as a notebook computer in an open state) andthe virtual object 1610 (shown as a virtual notebook computer in theopen state). For illustration purposes, the virtual object 1610 (i.e.,the virtual notebook computer) emulates or copies the real object 1620(i.e., the real notebook computer). As such, the virtual object can looklike or be similar to the real object. For instance, the virtual objectappears as a same make and model notebook computer as the real notebookcomputer. Alternatively, the virtual object can be different than ordissimilar to the real object.

The virtual object 1610 is placed or positioned on, with, over, next to,or adjacent to the real object 1620 and maintains a consistent positionwith respect to the real object 1620 while the user 1630 moves his headand changes the field of view 1640. For instance, when the user jigglesor moves his head, spacing between the virtual object 1610 and the realobject 1620 remains constant. Additionally, when the user jiggles ormoves his head, the virtual object 1610 does not appear to move withrespect to the real object 1620. Both the real object 1620 and thevirtual object 1610 remain stationary while the head 1630 of the usermoves in different directions.

FIG. 16A illustrates the user 1630 looking straight toward the realobject 1620 and the virtual object 1610. From a point of view of theuser, the virtual object 1610 is next to on a left side of the realobject 1620.

FIG. 16B illustrates the user 1630 looking toward his right side butstill maintaining the real object 1620 and the virtual object 1610 inhis field of view 1640. As the user 1630 moves his head toward his rightside, the field of view 1640 also moves to this side. During thismovement, the wearable electronic device 1600 simultaneously compensatesand moves a position of the virtual object 1610 being displayed to theuser. Movement of the virtual object 1610 coincides with and negatesmovement of the wearable electronic device 1600 such that the virtualobject 1610 appears to remain motionless or still with respect to thereal object. For instance, as the user 1630 moves his head toward hisright side, the virtual object 1610 simultaneously moves with a samespeed toward the opposite side of the field of view 1640. In otherwords, the virtual object 1610 moves in an opposite direction but with asame speed as movement of the field of view 1640. This offsetting motionenables the virtual object 1610 to appear to remain still while thefield of view 1640 changes.

FIG. 16C illustrates the user 1630 looking toward his left side butstill maintaining the real object 1620 and the virtual object 1610 inhis field of view 1640. As the user 1630 moves his head toward his leftside, the field of view 1640 also moves to this side. During thismovement, the wearable electronic device 1600 simultaneously compensatesand moves a position of the virtual object 1610 being displayed to theuser. Movement of the virtual object 1610 coincides with and negatesmovement of the wearable electronic device 1600 such that the virtualobject 1610 appears to remain motionless or still with respect to thereal object. For instance, as the user 1630 moves his head toward hisleft side, the virtual object 1610 simultaneously moves with a samespeed but opposite direction toward the right side of the field of view1640.

FIGS. 16A-16C illustrate that as the user moves his head in differentdirections, the virtual object simultaneously moves to compensate andoffset this movement such that the virtual object maintains a consistentposition with respect to the real object.

Consider an example in which the wearable electronic device of the userexecutes instructions that assist the user in solving a problem withanother electronic device that sits on a table in front of the user.While the user looks at this electronic device, a virtual image of thiselectronic device appears to sit on the table next to the realelectronic device. These two objects are situated in a center of thefield of view of the user. The virtual object then begins to move and/orotherwise instruct the user about this electronic device. During thisinstruction, the user moves his head and looks away (such as looking tohis side). During this movement, the real electronic device does notmove since it remains stationary on the table. Likewise, the virtualimage of the electronic device also appears to remain stationary on thetable next to the real electronic device. In order to make thisappearance, the wearable electronic device changes a position on itslens or display of the virtual image of the electronic device. In otherwords, a projection, image, or location of the virtual electronic devicechanges on the display or lens in order to compensate for the movementof the field of view of the user. This change corresponds to themovement of the field of view such that the virtual image appears toremain stationary with respect to the real object while both objectssimultaneously remain in the field of view.

A consistent distance can also be maintained between the virtual andreal objects while the real object moves in the field of view of theuser. For example, while the virtual object is positioned next to thereal object, a wearer of the wearable electronic device picks up thereal object and moves it to another location in the field of view. Inresponse to this motion, the virtual object simultaneously moves toappear to maintain a consistent distance with the real object. Thus thevirtual object tracks or follows the movements of the real object so thevirtual object remains positioned next to the real object. These twoobjects move together in conjunction or in tandem.

FIGS. 17A-17D show a wearable electronic device 1700 that provides avirtual object 1710 with a real object 1720. For illustration, a user1730 wears the wearable electronic device 1700 (shown by way of exampleas electronic glasses or electronic eyewear). The user 1730 has a fieldof view 1740 through the wearable electronic device 1700 with aperiphery, edge, or perimeter shown with dashed lines 1750.

The field of view 1740 of the user 1730 includes the real object 1720(shown by way of example as a notebook computer in an open state) andthe virtual object 1710 (shown as a virtual notebook computer in theopen state). For illustration purposes, the virtual object 1710 (i.e.,the virtual notebook computer) emulates or copies the real object 1720(i.e., the real notebook computer). As such, the virtual object can looklike or be similar to the real object. For instance, the virtual objectappears as a same make and model notebook computer as the real notebookcomputer. Alternatively, the virtual object can be different than ordissimilar to the real object.

The virtual object 1710 is placed or positioned on, with, over, next to,or adjacent to the real object 1720 and moves to different positionswith respect to the real object 1720 to stay within the field of view1740 while the user 1730 moves his head and changes the field of view1740. For instance, when the user moves his head, this movement cancause a location of the virtual object 1710 to be out of the field ofview 1740 and hence no longer visible to the user 1730. In response tothis movement, a position of the virtual object 1710 in the field ofview 1740 changes in order to maintain the virtual object in the fieldof view 1740 and with the real object 1720.

The virtual object 1710 transitions or moves to a new location withinthe field of view 1740 when movement of the wearable electronic device1700 causes the virtual object 1710 to no longer be visible or presentwithin the field of view 1740. For example, when the user 1730 moves hishead to one side, this movement can cause a current location of thevirtual object 1710 to be out of the field of view 1740 while the realobject 1720 remains in the field of view 1740. To compensate or adjustfor this movement, a position or location of the virtual object 1710 inthe field of view 1740 changes so the virtual object 1710 remains in thefield of view 1740 and with or on the real object 1720.

In an example embodiment, spacing between the virtual object 1710 andthe real object 1720 remains constant or unchanged until a view of thevirtual object 1710 becomes obstructed or until movement of the field ofview is significant enough to remove the virtual object 1710 or aportion of the virtual object 1710 from the field of view. For example,when the user jiggles or slightly moves his head to one side, thevirtual object does 1710 does not appear to move with respect to thereal object 1720 if both the virtual object 1710 and the real object1720 remain unobstructed or in the field of view 1740.

Both the real object 1720 and the virtual object 1710 remain stationarywhile the head 1730 of the user moves in different directions withslight movements. When these movements, however, are significant enoughto impede or restrict a view of the virtual object 1710 (such as pushingthe virtual object 1710 or portions thereof out of the field of view1740), the virtual object 1710 is repositioned within the field of view1740 to remain visible to the user 1730. Additionally, the virtualobject 1710 is repositioned within the field of view 1740 when its viewis obstructed. For example, while the virtual and real objects arewithin the field of view, another object impedes, restricts, orobstructs the user's view of the virtual object. In this instance, thevirtual object is repositioned within the field of view to provide theuser with a clear or unobstructed view of the virtual object. As anotherexample, the virtual object is repositioned in the field of view when anedge or perimeter of the field of view hits or touches the virtualobject.

FIG. 17A illustrates the user 1730 looking straight toward the realobject 1720 and the virtual object 1710. From a point of view of theuser, the virtual object 1710 is next to and on a left side of the realobject 1720.

FIG. 17B illustrates the user 1730 looking toward his right side butstill maintaining the real object 1720 in his field of view 1740. If aposition of the virtual object 1710 were not moved, then the virtualobject 1710 would no longer remain in the field of view of the user. Theposition of the field of view from FIG. 17A is shown with dashed lines1780, and the position of the virtual object from FIG. 17A is shown withdashed lines 1782. If the virtual object did not move with respect tothe real object 1720, then the virtual object would no longer be in thefield of view 1740 of the user shown in FIG. 17B. Arrow 1784 shows thatthe virtual object 1710 moved from its previous position 1782 in thefield of view to a different position in the field of view.Specifically, the virtual object 1710 went from being positioned on aleft side of the real object (shown in FIG. 17A) to being positioned ona right side of the real object (shown in FIG. 17B).

While the virtual object moves to different positions within the fieldof view or different positions with respect to the real object, adistance between the virtual object and real object remains consistent.For example, FIG. 17A shows a distance D exists between the virtualobject 1710 and the real object 1720 while the virtual object ispositioned on a left side of the real object. In response to the user1730 moving his head to his right, the virtual object 1710 moves orswitches to a new location with respect to the real object. FIG. 17Bshows the virtual object 1710 moved to a different location with respectto the real object 1720 (the virtual object is now positioned on a rightside of the real object). A relative distance between the virtual objectand the real object did not change after the virtual object moved orswitched to a different location. FIG. 17B shows a distance D existsbetween the virtual object 1710 and the real object (an equivalentdistance D shown in FIG. 17A).

FIGS. 17A and 17C illustrate that a distance between the virtual objectand the real object can vary or change in response to movement of thewearable electronic device. FIG. 17A shows a distance D between thevirtual object 1710 and the real object 1720. FIG. 17C shows a distanceD1 between the virtual object 1710 and the real object 1720 in which D1is less than D.

As the user 1730 moves his head toward his right, an amount of availablespace 1786 between the virtual object 1710 and edge or side 1788 of thefield of view 1740 reduces or lessens. The virtual object 1710 canremain in this space 1786 and to the left of the real object 1720 aslong as the available space 1786 is large enough or sufficient enough inshape and/or size to accommodate a shape and/or size of the virtualobject 1710. When the size and/or shape of this available space 1786 isnot sufficient, then the virtual object 1710 moves to a new location inthe field of view. For example, the virtual object 1710 moves closer tothe real object 1720, and this movement reduces a distance between thevirtual object and the real object. Moving closer to the real objectincreases the available space 1786 by reducing the distance between thevirtual object and the real object. This movement also causes thevirtual object to move farther away from edge 1788. As shown in FIGS.17A and 17C, the distance D1 is less than the distance D since thevirtual object 1710 moved closer to the real object 1720 to compensatefor movement of the wearable electronic device 1700 and correspondingreduction in available space 1786 for the virtual object to exist at itscurrent location with respect to the real object.

FIGS. 17A and 17D illustrate that a size and/or shape of the virtualobject can vary or change in response to movement of the wearableelectronic device. FIG. 17A shows the virtual object 1710 and the realobject 1720 having a similar or equivalent size and shape. FIG. 17Dshows the virtual object 1710D having a reduced size in comparison tothe virtual object 1710 shown in FIG. 17A.

As the user 1730 moves his head toward his right, an amount of availablespace 1786 between the virtual object 1710 and edge or side 1788 of thefield of view 1740 reduces or lessens. The virtual object 1710 canremain in this space 1786 and to the left of the real object 1720 aslong as the available space 1786 is large enough or sufficient enough inshape and/or size to accommodate a shape and/or size of the virtualobject 1710. When the size and/or shape of this available space 1786 arenot sufficient, then the virtual object 1710 changes its size and/orshape to compensate for the reduction in size of available space 1786.For example, the virtual object becomes smaller as the edge 1788impinges on or approaches a boundary of the virtual object. Reducing asize and/or changing a shape of the virtual object increases theavailable space 1786 by reducing an amount of space needed or requiredto maintain the virtual object at its current location. As shown inFIGS. 17A and 17D, a size of the virtual object 1710 in FIG. 17A isgreater than a size of the virtual object 1710D in FIG. 17D. Thisreduction in size compensates for movement of the wearable electronicdevice 1700 and corresponding reduction in available space for thevirtual object to exist at its current location with respect to the realobject.

As the user moves his head or a position of the wearable electronicdevice changes, a size of the virtual object, a shape of the virtualobject, and/or a distance between the virtual object and the real objectchanges or adjusts to offset or accommodate the movement of the head orwearable electronic device. Such adjustments enable the virtual objectto remain in a steady or consistent position with respect to the realobject in spite of movement to the field of view.

To compensate or correct for such movement, the virtual object can alsochanges its orientation and/or its view. For example, if the virtualobject exists as a 3D image, then the virtual object rotates about oneof its axes (such as rotating about one or more of an x-axis, a y-axis,or a z-axis). As another example, the virtual object changes from beingpresented as a 3D object to being presented as a 2D object. As yetanother example, the virtual object changes its view to the user (suchas changing from being presented in a perspective view to beingpresented in a plan view).

Consider an example in which the wearable electronic device moves, andthis movement causes an edge of the field of view to touch or overlap onthe virtual object. Without an adjustment to the virtual object, aportion of the virtual object is no longer visible to the user. Inresponse to this movement, a distance between the virtual object and thereal object decreases. This distance can decrease to zero. If the edgeof the field of view continues to impinge or cover the virtual objectafter the distance is zero (i.e., the virtual and real object are rightnext to each other with little or no space between them), then thevirtual object can continue to move to a position that is over orsuperimposed on top of the real object (for example, see FIG. 14).

FIGS. 18A-18C show a wearable electronic device 1800 that translateswriting on a real object 1810. For illustration, a user 1820 wears thewearable electronic device 1800 (shown by way of example as electronicglasses or electronic eyewear). The user 1820 has a field of view 1830through the wearable electronic device 1800 with a periphery, edge, orperimeter shown with dashed lines 1840.

The real object 1810 includes one side with text or writing 1850 (shownas writing in English that says “My name is Philip”). The user viewsthis writing in its original language (i.e., English) on the real object1810.

FIG. 18B shows a virtual object 1860 superimposed over the writing 1850on the real object 1810. This virtual object 1860 is a translation ofthe writing in the English language to Thai language (shown as atranslation in Thai of the English phrase “My name is Philip”). Thevirtual object 1860 of the Thai language is on or over the originalwriting in English such that the real object 1810 appears to the user1820 through the wearable electronic device 1800 to be written in Thai,not English.

FIG. 18C shows the virtual object 1860 superimposed over writing on thereal object 1810 as the user 1820 moves his head. As the user 1820 moveshis head in different directions, the virtual object 1860 remainsstationary on the real object 1810 such that the writing on the realobject appears to be in Thai to the user. As shown in FIG. 18C, the user1820 moves his head to his right side, yet the translation of theEnglish writing to Thai remains on the real object.

FIG. 19 shows a computer system or electronic device system 1900 thatincludes wearable electronic devices 1910A and 1910B that communicatewith each other over a network 1920. For illustration, a user 1930Awears the wearable electronic device 1910A, and a user 1930B wears thewearable electronic device 19106 (both devices shown by way of exampleas electronic glasses or electronic eyewear). Users 1930A and 19306 haverespective fields of view 1940A and 19406 with respective peripheries,edges, or perimeters shown with dashed lines 1950A and 1950B.

The field of view 19406 of user 19306 includes a real object 19606, andthe field of view 1940A of user 1930A includes an image of the realobject 1960A (such as a photo or a video of the real object 1960B). Thisimage of the real object 1960A appears to the user 1930A in real time asthe wearable electronic device 19106 captures this image. The field ofview 19406 of user 19306 also includes a virtual object 19706, and thefield of view 1940A of user 1930A also includes a virtual object 1970A.For illustration, the real object 1960B and the image of the real object1960A are shown as a notebook computer, and the virtual objects 1970Aand 19706 are shown as virtual images of the notebook computer.

The wearable electronic devices 1910A and 1910B communicate over thenetwork 1920 with other electronic devices 1980 (such as one or moreservers). These electronic devices 1980 include one or more ofprocessors 1982, memories 1984, a modeling and diagnosis system 1986,and other components discussed herein.

Actions on or movements of the virtual object 1970A simultaneously occuras actions on or movements of the virtual object 19706. For example,when a manipulation occurs on the virtual object 1970A shown to user1930A, this manipulation simultaneously occurs on the virtual object1970B shown to user 1930B.

Consider an example in which user 1930B has a software issue or problemwith his notebook computer 19606 and seeks assistance in resolving thisissue or problem from user 1930A. User 1930B views his notebook computer1960B with his wearable electronic device 19106, and a real time videoimage of his notebook computer appears to user 1930A as the notebookcomputer 1960A. Computer software (such as programs and applications)executing on the notebook computer 19606 appear as virtual executions orsimulations on the virtual notebook computer 1970A. As such, user 1930Ahas a view of the real notebook computer 1960A and a view of a virtualnotebook computer 1970A that emulates or simulates the real notebookcomputer 19606 (including emulation or simulation of hardware, software,system state, etc.). For example, the virtual notebook computer 1970Auses the modeling and diagnosis system 1986 to replicate a current stateof the real notebook computer 19606. The user 1930A performs physicalmanipulations or actions on or to the virtual notebook computer 1970A,and these manipulations or actions replicate on the virtual notebookcomputer 1970B shown to user 1930B. In this manner, the user 1930B cansee with the wearable electronic device 19106 what instructions oractions the user 1930A desires the user 1930B to perform on the realnotebook computer 19606. When the user 19306 performs the instructionsor actions, the user 1930A sees this performance on the image of thenotebook computer 1960A and/or the virtual notebook computer 1970A.

Consider the example above in which user 19306 has a software issue orproblem with his notebook computer 19606 and seeks assistance inresolving this issue or problem from user 1930A. User 1930A uses a mouseor keyboard to provide instructions by navigating a cursor that appearson the display of virtual notebook computer 1970A. Navigation of thiscursor also appears on the virtual notebook computer 1970B. The user1930B, wearable electronic device 1910B, or a software program thenexecutes these instructions on the real notebook computer 1960B (such asa software program automatically executing the instructions on the realnotebook computer 19606 or the user 19306 interacting with his mouse orkeyboard to emulate navigation of his cursor to match the instructionsreceived from user 1930A).

Consider the example above in which user 19306 has a software issue orproblem with his notebook computer 19606 and seeks assistance inresolving this issue or problem from user 1930A. User 1930A types intohis keyboard and performs a series of commands that execute on thevirtual notebook computer 1970A. The virtual notebook computer 1970Aemulates software, hardware, and current state information on the realnotebook computer 1960B. These commands appear to resolve the issue orproblem on the virtual notebook computer 1970A so the user 1930Aauthorizes these commands to execute on the real notebook computer19606. User 1930A watches the real notebook computer 1960A and visuallyverifies that the commands resolved the issue or problem. The user 1930Awas thus able to resolve the issue with the virtual notebook computer1970A and then see the problem being cured on the real notebook computer1960B (i.e., image 1960A presents the user 1930A with a real-time viewof the real notebook computer 1960B).

Consider the example above in which user 19306 has a software issue orproblem with his notebook computer 19606 and seeks assistance inresolving this issue or problem from user 1930A. User 1930A instructsuser 1930B to simultaneously hit or press the command-option-esc buttonson the keyboard of the real notebook computer 1960B. User 1930B,however, is unsure where these buttons exist on his keyboard andrequests further assistance. In response to this request, user 1930Areaches his hand into the field of view 1940A and points to and/orpresses the command-option-esc buttons on the virtual notebook computer1970A. A virtual image of the user's hand appears in the field of view1940B, and user 1930B sees the hand of user 1930A hitting thecommand-option-esc buttons on the virtual notebook computer 1970B. Afterseeing this demonstration, user 19306 knows the location of thesebuttons and presses the command-option-esc buttons on the real notebookcomputer 19606.

FIG. 20 shows a computer system or electronic device system 2000 thatincludes wearable electronic devices 2010A and 20106 that communicatewith each other over a network 2020. For illustration, a user 2030Awears the wearable electronic device 2010A, and a user 2030B wears thewearable electronic device 20106 (both devices are shown by way ofexample as electronic glasses or electronic eyewear). Users 2030A and2030B have respective fields of view 2040A and 2040B with respectiveperipheries, edges, or perimeters shown with dashed lines 2050A and2050B.

The field of view 2040B of user 2030B includes a real object 2060 and avirtual object 2070B, and the field of view 2040A of user 2030A includesa virtual object 2070A. The virtual objects 2070A and 2070B are virtualimages or copies of the real object 2060. For illustration, the realobject 2060 is shown as a plate or cover with four screws, and thevirtual objects 2070A and 2070B are virtual images of this plate orcover.

Actions on or movements of the virtual object 2070A simultaneously occuras actions on or movements of the virtual object 2070B. These actions ormovements on the virtual object 2070B provide assistance or instructionsto the user 2030B on how to manipulate or act on the real object 2060.

Consider an example in which the user 2030A assists the user 2030B at aremote location in removing screws from the real cover 2060. User 2030Agrabs a screwdriver 2080 (which is a real object) and moves thisscrewdriver toward a point in space where a virtual screw 2082A appearson the virtual cover 2070A in his field of view 2040A. Simultaneously, avirtual image of the screwdriver 2086 appears moving toward a point inspace where a virtual screw 2082B appears on the virtual cover 2070B inthe field of view 2040B of user 2030B (the hand and arm of user 2030Aholding the screwdriver can also appear in the field of view 2040B butare emitted for ease of illustration). In response to thisvisualization, user 2030B grabs a real screwdriver (not shown) andengages it with the real screw 2090 on the real cover 2060.

Consider the example in which the user 2030A assists the user 2030B at aremote location in removing screws from the real cover 2060. When theuser 2030A moves the screwdriver 2080 to the location in space of thevirtual screw 2082A, the virtual screw 2082A visually changes or anindication is provided in order to signify its selection. For example,the virtual screw 2082A changes color, becomes highlighted, becomesbrighter, or undergoes a change to visually signify its selection by theuser 2030A. As other examples, an arrow, pointer, text, or other indiciaappear in the field of view near the virtual screw 2082A to visuallysignify its selection by the user 2030A.

Thus, user 2030A is able to use a real object 2080 to perform actions inspace on virtual objects 2070A. The real object 2080 and the virtualobject 2070A appear as virtual objects to another user 2030B. This user2030B views the actions from the user 2030A and performs the actions ona real object 2060.

FIG. 21 shows a computer system or electronic device system 2100 thatincludes wearable electronic devices 2110A, 21106, and 2110C thatcommunicate with each other over a network 2120. For illustration, auser 2130A wears the wearable electronic device 2110A; a user 21306wears the wearable electronic device 2110B; and a user 2130C wears thewearable electronic device 2110C (these three devices are shown by wayof example as electronic glasses or electronic eyewear). Users 2130A,21306, and 2130C have respective fields of view 2140A, 2140B, and 2140Cwith respective peripheries, edges, or perimeters shown with dashedlines 2150A, 2150B, and 2150C.

The fields of view 2140A, 2140B, and 2140C each include a virtual object2160 seen from a different point of view. User 2130A sees the virtualobject 2160 as a 3D virtual box on a real table 2170; user 2130B seesthe virtual object 2160 as a one side of the virtual box on the realtable 2170; and user 2130C sees the virtual object 2160 as a top of thevirtual box on the real table 2070. Each of the users sees the virtualobject 2160 from a different location and hence from a differentpoint-of-view. User 2130A sees the virtual box from an elevatedpoint-of-view; user 21306 sees the virtual box from a sidepoint-of-view; and user 2130C sees the virtual box from a toppoint-of-view.

Consider an example in which the virtual object 2160 is a virtual boxthat appears to sit on top of real table 2170, and users 2130A, 2130B,and 2130C are situated at different physical locations in a room withthe table. A view of the box depends on the physical location of theuser with respect to the box. For example, a user situated close to andbelow the box will have a different point-of-view and hence differentview of the box than a user situated above and farther away from thebox.

The virtual object appears to the users through their respectivewearable electronic devices (such as being displayed on displays or lensof these respective electronic devices). A user without a wearableelectronic device would not be able to see the virtual object since itis projected with or displayed on the wearable electronic device.

One or more electronic devices create the virtual object or objects thatare presented or displayed to users with the wearable electronicdevices. For example, users look through a transparent screen or windowto see a 3D virtual object. These electronic devices and/or a computersystem include one or more of a gyroscope, an accelerometer, an antenna(such as a WiFi antenna), a motion sensor and/or motion detector, acamera, a projector, a position sensor and/or position tracker, facialrecognizer, sensor, lens, and mirror.

In example embodiments, the wearable electronic device can generate,provide, project, or display the virtual object to the user. As oneexample, the wearable electronic device generates or assists ingenerating the virtual object. As another example, another electronicdevice generates the virtual object that is seen through or with thewearable electronic device.

After a virtual object is generated, users can view the virtual objectfrom different viewpoints as if the virtual object were a real object.Users can also interacts with the virtual object, such as addingcomponents to the virtual object, deleting components from the virtualobject, scaling portions of the virtual object, changing colors, size,and/or shape of the virtual object, etc. While viewing and/orinteracting with the virtual object, users can see each other andinteract with the real world.

FIG. 22 is a computer system or electronic device system 2200 inaccordance with an example embodiment. The computer system 2200 includesone or more electronic devices 2210 that include components of computerreadable medium (CRM) or memory 2215, a display 2220, a processing unit2225, one or more interfaces 2230 (such as a network interface, agraphical user interface, a natural language user interface, a naturaluser interface, a reality user interface, a kinetic user interface, anaugmented reality user interface, and/or an interface that combinesreality and virtuality), a camera 2235, one or more sensors 2240 (suchas micro-electro-mechanical systems sensor, an optical sensor,radio-frequency identification sensor, a global positioning satellitesensor, a solid state compass, gyroscope, and/or an accelerometer), anda recognition system 2245 (such as speech recognition system or agesture recognition system). The sensors can further include motiondetectors (such as sensors that detect motion with one or more ofinfrared, optics, radio frequency energy, sound, vibration, andmagnetism).

The computer system 2200 further includes a wearable electronic device2250 (including a processing unit 2251, a memory 2252, a display 2253, acamera 2254, an eye tracking device or eye tracker 2255, and one or moreinterfaces 2256).

The computer system 2200 further includes a pair of wearable electronicglasses 2260 (including a processing unit 2261, a memory 2262, amagnetometer 2263, a camera 2264, a touchpad 2265, an optical headmounted display 2266, a gyroscope 2267, an accelerometer 2268, a lightsensor 2269, and one or more interfaces 2270), and an imagery system2280 (such as an optical projection system, a virtual image displaysystem, virtual augmented reality system, and/or a spatial augmentedreality system). By way of example, the augmented reality system usesone or more of image registration, computer vision, and/or videotracking to supplement and/or change real objects and/or a view of thephysical, real world.

The electronic device 2210, wearable electronic device 2250, wearableelectronic glasses 2260, and imagery system 2280 communicate with eachother and other electronic devices through one or more networks 2290.

FIG. 22 shows various components in a single electronic device and othercomponents distributed throughout the system. Alternatively, one or moreof these components can be distributed or included in various electronicdevices, such as some components being included in an HPED, somecomponents being included in a server, some components being included instorage accessible over the Internet, some components being in animagery system, some components being in wearable electronic devices,and some components being in various different electronic devices thatare spread across a network or a cloud, etc.

The processor unit includes a processor (such as a central processingunit, CPU, microprocessor, application-specific integrated circuit(ASIC), etc.) for controlling the overall operation of memory (such asrandom access memory (RAM) for temporary data storage, read only memory(ROM) for permanent data storage, and firmware). The processing unitcommunicates with memory and performs operations and tasks thatimplement one or more blocks of the flow diagrams discussed herein. Thememory, for example, stores applications, data, programs, algorithms(including software to implement or assist in implementing exampleembodiments) and other data.

Blocks and/or methods discussed herein can be executed and/or made by auser, a user agent of a user, a software application, an electronicdevice, a computer, a computer system, and/or an intelligent personalassistant.

As used herein, “augmented reality” is a view of the real, physicalworld in which elements are augmented or modified with computer orprocessor generated input, such as sound, graphics, GPS data, video,and/or images. Virtual images and objects can be overlaid on the realworld that becomes interactive with users and digitally manipulative.

As used herein, “field of view” or “field of vision” is the extent ofthe observable world that is seen at a given moment. For example,without mechanical assistance, humans have almost one hundred and eighty(180) degrees of forward-facing field of view with about one hundred andtwenty (120) degrees of this field being binocular vision.

As used herein, a “virtual image” or “virtual object” is computer orprocessor generated image or object. This image or object often appearsto a user in the real, physical world (such as a virtual 3D dimensionalobject that the user views in the real world).

As used herein, a “wearable electronic device” is a portable electronicdevice that is worn on or attached to a person. Examples of such devicesinclude, but are not limited to, electronic watches, electronicnecklaces, electronic clothing, head-mounted displays, electroniceyeglasses or eye wear (such as glasses in which augmented realityimagery is projected through or reflected off a surface of a lens),electronic contact lenses (such as bionic contact lenses that enableaugmented reality imagery), an eyetap, handheld displays that affix to ahand or wrist or arm (such as a handheld display with augmented realityimagery), and HPEDs that attach to or affix to a person.

In some example embodiments, the methods illustrated herein and data andinstructions associated therewith are stored in respective storagedevices, which are implemented as computer-readable and/ormachine-readable storage media, physical or tangible media, and/ornon-transitory storage media. These storage media include differentforms of memory including semiconductor memory devices such as DRAM, orSRAM, Erasable and Programmable Read-Only Memories (EPROMs),Electrically Erasable and Programmable Read-Only Memories (EEPROMs) andflash memories; magnetic disks such as fixed, floppy and removabledisks; other magnetic media including tape; optical media such asCompact Disks (CDs) or Digital Versatile Disks (DVDs). Note that theinstructions of the software discussed above can be provided oncomputer-readable or machine-readable storage medium, or alternatively,can be provided on multiple computer-readable or machine-readablestorage media distributed in a large system having possibly pluralnodes. Such computer-readable or machine-readable medium or media is(are) considered to be part of an article (or article of manufacture).An article or article of manufacture can refer to any manufacturedsingle component or multiple components.

Method blocks discussed herein can be automated and executed by acomputer, computer system, user agent, and/or electronic device. Theterm “automated” means controlled operation of an apparatus, system,and/or process using computers and/or mechanical/electrical deviceswithout the necessity of human intervention, observation, effort, and/ordecision.

The methods in accordance with example embodiments are provided asexamples, and examples from one method should not be construed to limitexamples from another method. Further, methods discussed withindifferent figures can be added to or exchanged with methods in otherfigures. Further yet, specific numerical data values (such as specificquantities, numbers, categories, etc.) or other specific informationshould be interpreted as illustrative for discussing exampleembodiments. Such specific information is not provided to limit exampleembodiments.

What is claimed is:
 1. A method, comprising: displaying, on a display ofwearable electronic glasses, a virtual image of a real object next tothe real object that is in a field of view of the wearable electronicglasses; displaying, on the display of the wearable electronic glassesand while the virtual image of the real object is next to the realobject, movement of the virtual image of the real object to show a taskto be completed on the real object; detecting, with the wearableelectronic glasses and while the virtual image of the real object isnext to the real object, completion of the task on the real object suchthat the movement of the real object emulates the movement of thevirtual image of the real object; and switching the virtual image of thereal object from being located on one side of the real object to beinglocated on an opposite side of the real object when the wearableelectronic glasses move to have the field of view that obstructs a viewof the virtual image of the real object on the one side.
 2. The methodof claim 1 further comprising: maintaining a consistent distance betweenthe real object and the virtual image of the real object being displayedon the display while the wearable electronic glasses move and change alocation of where the real object exists in a field of view of thewearable electronic glasses.
 3. The method of claim 1 furthercomprising: translating text of a first language that is located on asurface of the real object into a second language; and displaying thetext in the second language on a surface of the virtual image of thereal object such that the virtual image of the real object emulates anappearance of the real object but has text the being displayed in thesecond language.
 4. The method of claim 1 further comprising:displaying, on a display of a handheld portable electronic device,controls that provide a wearer of the wearable electronic glasses withinstructions regarding which direction to move and how fast to move inthe direction; and displaying, on the display of the wearable electronicglasses, the controls that provide the wearer of the wearable electronicglasses with the instructions regarding which direction to move and howfast to move in the direction.
 5. The method of claim 1 furthercomprising: displaying, on a display of other wearable electronicglasses that are remote from the wearable electronic glasses, video ofthe real object; detecting, at the other wearable electronic glasses,movements of a hand of a wearer of the other wearable electronic glassestoward the real object; and displaying, on the display of the wearableelectronic glasses, a virtual hand that emulates the movements of thehand toward the real object.
 6. The method of claim 1 furthercomprising: displaying the virtual image of the real object superimposedover the real object; and moving, while the virtual image of the realobject is superimposed over the real object, a portion of the virtualimage of the real object to show the task to be completed on the realobject.
 7. Wearable electronic glasses, comprising: a display; a memorythat stores instructions; a position sensor; and a processor incommunication with the display, with the position sensor, and with thememory to execute the instructions to: detect a real object that isvisible through the wearable electronic glasses; display, on the displayand concurrently with the real object, a virtual object that emulatesthe real object; display, on the display and while the virtual object isconcurrently displayed with the real object, movement of the virtualobject that shows an action to be performed on the real object; detect,with the position sensor and while the virtual object is concurrentlydisplayed with the real object, movement of the real object such thatthe movement of the real object copies the movement of the virtualobject; display, on the display, the virtual object adjacent to one sideof the real object in a field of view of the wearable electronicglasses; detect movement of the wearable electronic glasses in which thevirtual object is no longer within the field of view of the wearableelectronic glasses; and move, in response to the virtual object nolonger being within the field of view, the virtual object to an oppositeside of the real object such that the virtual object is within the fieldof view of the wearable electronic glasses.
 8. The wearable electronicglasses of claim 7, wherein the processor further executes theinstructions to: track movement of eyes of a wearer of the wearableelectronic glasses; and activate the virtual content to display on thedisplay of the wearable electronic glasses when the eyes of the wearerfocus on the real object.
 9. The wearable electronic glasses of claim 7,wherein the processor further executes the instructions to: receive,from a remote pair of wearable electronic glasses, images of the virtualobject that instruct a wearer of the wearable electronic glasses theaction to be performed on the real object.
 10. The wearable electronicglasses of claim 7, wherein the processor further executes theinstructions to: maintain a constant distance between the virtual objectbeing displayed on the display and the real object in a field of view ofa wearer of the wearable electronic glasses while the wearableelectronic glasses and the field of view moves and changes.
 11. Thewearable electronic glasses of claim 7, wherein the processor furtherexecutes the instructions to: receive, from a remote handheld portableelectronic device, instructions; and display, on the display and to awearer of the wearable electronic glasses, the instructions thatsimultaneously instruct which direction to move a head of the weareralong with which direction to move a body of the wearer along with whatspeed the wearer should proceed in the direction to move the body. 12.The wearable electronic glasses of claim 7, wherein the processorfurther executes the instructions to: receive, from a remote pair ofwearable electronic glasses, the virtual object that shows the action tobe performed on the real object, wherein the virtual object is a virtualhand of a wearer of the remote pair of the wearable electronic glasses.13. A non-transitory computer readable storage medium storinginstructions that cause a computer system to execute a method,comprising: display, through wearable electronic glasses andconcurrently with a real object, a virtual object that emulates the realobject in a field of view of the wearable electronic glasses; display,through the wearable electronic glasses and while the virtual object isconcurrently displayed with the real object, a change to the virtualobject that shows an action to be performed on the real object; capture,with the wearable electronic glasses and while the virtual object isconcurrently displayed with the real object, performance of the actionon the real object such that the performance of the action on the realaction copies the action of the virtual object; detect when the virtualobject begins to move outside a field of view of the wearable electronicglasses; and move, in response to detecting the virtual object movingoutside the field of view, the virtual object to a different location onan opposite side of the real object such that the different location onthe opposite side situates the virtual object in the field of view ofthe wearable electronic glasses.
 14. The non-transitory computerreadable storage medium storing instructions of claim 13 further tocause the computer system to execute the method comprising: display,through the wearable electronic glasses, a visual notification thatsignifies the action on the real objected captured by the wearableelectronic glasses did copy the action of the virtual object displayedthrough the wearable electronic glasses.
 15. The non-transitory computerreadable storage medium storing instructions of claim 13 further tocause the computer system to execute the method comprising: display,through the wearable electronic glasses, the virtual object at adistance from the real object while the real object is visible throughthe lens and within a field of view of the wearable electronic glasses;maintain the distance between the virtual object and the real object asconstant while the wearable electronic glasses move and change the fieldof view; and maintain the distance between the virtual object and thereal object as constant while the real object moves within the field ofview.
 16. The non-transitory computer readable storage medium storinginstructions of claim 13, wherein the virtual object emulates the realobject such that the virtual object has a size, shape, and color thatmatches a size, shape, and color of the real object.
 17. Thenon-transitory computer readable storage medium storing instructions ofclaim 13 further to cause the computer system to execute the methodcomprising: control, from a handheld portable electronic device that isremote from the wearable electronic glasses, movements of the virtualobject in real-time as the virtual object moves and is displayed throughthe wearable electronic glasses.